Skip to main content

Installing gluster in AWS EKS

This article is a continuance of [1]. Purpose of this article is to document the steps, issues and solutions to those issues we have to face when installing gluster in EKS (Elastic Kubernetes Service).

For gluster we need a disk to be attached with the K8s node. In EKS easiest way of implementing this is, adding it to the node configuration. So every time a node comes up, it comes up with a disk attached to the defined path. You can use this path in the topology.josn as mentioned in [1].

Next step is to install gluster using the gk-deploy script. The challenge comes here after. To use gluster in pods, you need to define a storage class. The heketi url mentioned in the storage class definition, should be accessible from master node. But the given heketi url is a cluster IP type k8s service. But in EKS deployments masters are managed by AWS and master don't have access to cluster IPs. So how we can solve this? Actually I tried to contact AWS support on this and I didn't got the answer from them. That lead to finding the answer by my own.

I created a LoadBalancer type k8s service in k8s fronting actual heketi service. First I started with external lb type but that let anyone outside the network access my heketi. Then I tried internal load balancer type. Then again I came to the original place. But this time I was able to make that internal lb type service accessible to the master by adding all my nodes to the same VPC as my master is in. And it allowed me to create the storage class without an issue and I could continue to create the PVC and use it in pods. Definition looks like below.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
  labels:
    glusterfs: heketi-deployment
    heketi: deployment
  name: heketi-external
  namespace: gluster
spec:
  externalTrafficPolicy: Cluster
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    glusterfs: heketi-pod
    heketi: pod
  sessionAffinity: None
  type: LoadBalancer

Hope this helps. Good luck on your deployment.

[1] https://wdfdo1986.blogspot.com/2019/05/consuming-file-system-artifacts-from.html
Labels:

Comments

Post a Comment

Popular posts from this blog

Generate JWT access tokens from WSO2 Identity Server

In Identity Server 5.2.0 we have created an interface to generate access tokens. Using that we have developed a sample to generate JWT tokens. You can find that sample under msf4j samples[1][2]. If you are build it as it is you will need to use Java 8 to build since msf4j is developed on Java 8. So you will need to run Identity Server on Java 8 as well. After building the project[2] please copy the jar inside target directory to $IS_HOME/repository/components/dropins/ directory. And then please add the following configuration to Identity.xml which is placed under $IS_HOME/repository/conf/identity/ folder inside tag OAuth . <IdentityOAuthTokenGenerator>com.wso2.jwt.token.builder.JWTAccessTokenBuilder</IdentityOAuthTokenGenerator> Then go to the database you used to store oauth tokens (This is the database pointed from the datasource you mentioned in the $IS_HOME/repository/conf/identity/identity.xml) and then alter the size of the column ACCESS_TOKEN of the tab...
6 comments
Read more

Integrate New Relic with WSO2 API Manager

In WSO2 API Manager, we have two transports. HTTP servlet transport and Passthru / NIO transport. All the web application requests are handled through HTTP servlet transport which is on 9763 port and 9443 port with ssl and here we are using tomcat inside WSO2 products. All the service requests are served via Passthru / NIO transport which is on 8082 and 8243 with ssl. When we integrate API Manager with new relic in the way discussed in blog posts [5],[6], new relic only detects the calls made to tomcat transports. So we couldn’t get the API calls related data OOTB. But by further analyzing new relic APIs I managed to find a workaround for this problem. New relic supports publishing custom events via their insights api[1]. So what we can do is publish these data via custom API handler[2]. Following is a sample implementation of a handler that I used to test the scenario. I will attach the full project herewith[7]. I have created an osgi bundle with this implementation so after building ...
Post a Comment
Read more

Setting up Single node Kubernetes Cluster with Core OS bare metal

You might know already there is an official documentation to follow to setup a Kubernetes cluster on Core OS bare metal. But when do that specially single node cluster, I found some gaps in that documentation [1] . And another reason for this blog post is to get everything into one place. So this blog post will describe how to overcome the issues of setting up a single node cluster. Installing Core OS bare metal. You can refer to doc [2]  to install core os.  First thing is about users. Documentation [2]  tells you how to create a user without password. To login as that user you will need ssh keys. So to create a user with username password, you can use a cloud-config.yaml file. Here is a sample. #cloud-config users: - name: user passwd: $6$SALT$3MUMz4cNIRjQ/Knnc3gXjJLV1vdwFs2nLvh//nGtEh/.li04NodZJSfnc4jeCVHd7kKHGnq5MsenN.tO6Z.Cj/ groups: - sudo - docker Here value for passwd is a hash value. One of the below methods can be used...
1 comment
Read more