Skip to main content

Installing gluster in AWS EKS

This article is a continuance of [1]. Purpose of this article is to document the steps, issues and solutions to those issues we have to face when installing gluster in EKS (Elastic Kubernetes Service).

For gluster we need a disk to be attached with the K8s node. In EKS easiest way of implementing this is, adding it to the node configuration. So every time a node comes up, it comes up with a disk attached to the defined path. You can use this path in the topology.josn as mentioned in [1].

Next step is to install gluster using the gk-deploy script. The challenge comes here after. To use gluster in pods, you need to define a storage class. The heketi url mentioned in the storage class definition, should be accessible from master node. But the given heketi url is a cluster IP type k8s service. But in EKS deployments masters are managed by AWS and master don't have access to cluster IPs. So how we can solve this? Actually I tried to contact AWS support on this and I didn't got the answer from them. That lead to finding the answer by my own.

I created a LoadBalancer type k8s service in k8s fronting actual heketi service. First I started with external lb type but that let anyone outside the network access my heketi. Then I tried internal load balancer type. Then again I came to the original place. But this time I was able to make that internal lb type service accessible to the master by adding all my nodes to the same VPC as my master is in. And it allowed me to create the storage class without an issue and I could continue to create the PVC and use it in pods. Definition looks like below.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
  labels:
    glusterfs: heketi-deployment
    heketi: deployment
  name: heketi-external
  namespace: gluster
spec:
  externalTrafficPolicy: Cluster
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    glusterfs: heketi-pod
    heketi: pod
  sessionAffinity: None
  type: LoadBalancer

Hope this helps. Good luck on your deployment.

[1] https://wdfdo1986.blogspot.com/2019/05/consuming-file-system-artifacts-from.html

Comments

Popular posts from this blog

Generate JWT access tokens from WSO2 Identity Server

In Identity Server 5.2.0 we have created an interface to generate access tokens. Using that we have developed a sample to generate JWT tokens. You can find that sample under msf4j samples[1][2]. If you are build it as it is you will need to use Java 8 to build since msf4j is developed on Java 8. So you will need to run Identity Server on Java 8 as well. After building the project[2] please copy the jar inside target directory to $IS_HOME/repository/components/dropins/ directory. And then please add the following configuration to Identity.xml which is placed under $IS_HOME/repository/conf/identity/ folder inside tag OAuth . <IdentityOAuthTokenGenerator>com.wso2.jwt.token.builder.JWTAccessTokenBuilder</IdentityOAuthTokenGenerator> Then go to the database you used to store oauth tokens (This is the database pointed from the datasource you mentioned in the $IS_HOME/repository/conf/identity/identity.xml) and then alter the size of the column ACCESS_TOKEN of the tab

Oauth custom basic authenticator with WSO2 IS 5.1.0

WSO2 Identity Server supports Oauth2 authorization code grant type with basic authentication OOTB. But basic authentication is done only with WSO2 user store. So there could be use cases that basic authentication has to be done against some other system. In this case you follow below steps to achieve your requirement. First you need to create an class which extends AbstractApplicationAuthenticator and implements LocalApplicationAuthenticator. Because this class is going to act as your application authenticator so it needs to be an implementation of application authenticator interface and to achieve this it needs to be a local authenticator as well. [2] public class CustomBasicAuthenticator extends AbstractApplicationAuthenticator implements LocalApplicationAuthenticator {   Then you need to override the initiateAuthenticationRequest method so you can redirect to the page to enter user and password. In my sample I redirected to the page that is used by our default basic au

Integrate New Relic with WSO2 API Manager

In WSO2 API Manager, we have two transports. HTTP servlet transport and Passthru / NIO transport. All the web application requests are handled through HTTP servlet transport which is on 9763 port and 9443 port with ssl and here we are using tomcat inside WSO2 products. All the service requests are served via Passthru / NIO transport which is on 8082 and 8243 with ssl. When we integrate API Manager with new relic in the way discussed in blog posts [5],[6], new relic only detects the calls made to tomcat transports. So we couldn’t get the API calls related data OOTB. But by further analyzing new relic APIs I managed to find a workaround for this problem. New relic supports publishing custom events via their insights api[1]. So what we can do is publish these data via custom API handler[2]. Following is a sample implementation of a handler that I used to test the scenario. I will attach the full project herewith[7]. I have created an osgi bundle with this implementation so after building