Added eksengine to engines/__init__.py by mikezuniga · Pull Request #315 · Mirantis/kqueen

mikezuniga · 2018-06-06T15:40:24Z

Added boto3 to setup.py requirements
Added eks engine
Added eks kubeconfig template
Docs added
TODO added

current issue related with kqueen-k8s-api methods:

api_1           | HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'Date': 'Sun, 10 Jun 2018 10:12:10 GMT', 'Content-Length': '234'})
api_1           | HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"services is forbidden: User \"system:anonymous\" cannot list services at the cluster scope","reason":"Forbidden","details":{"kind":"services"},"code":403}
api_1           |
api_1           | Traceback (most recent call last):
api_1           |   File "/code/kqueen/models.py", line 124, in status
api_1           |     'addons': kubernetes.list_services(filter_addons=True),
api_1           |   File "/code/kqueen/kubeapi.py", line 238, in list_services
api_1           |     include_uninitialized=include_uninitialized
api_1           |   File "/usr/local/lib/python3.6/site-packages/kubernetes/client/apis/core_v1_api.py", line 14358, in list_service_for_all_namespaces
api_1           |     (data) = self.list_service_for_all_namespaces_with_http_info(**kwargs)
api_1           |   File "/usr/local/lib/python3.6/site-packages/kubernetes/client/apis/core_v1_api.py", line 14455, in list_service_for_all_namespaces_with_http_info
api_1           |     collection_formats=collection_formats)
api_1           |   File "/usr/local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 321, in call_api
api_1           |     _return_http_data_only, collection_formats, _preload_content, _request_timeout)
api_1           |   File "/usr/local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 155, in __call_api
api_1           |     _request_timeout=_request_timeout)
api_1           |   File "/usr/local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 342, in request
api_1           |     headers=headers)
api_1           |   File "/usr/local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 231, in GET
api_1           |     query_params=query_params)
api_1           |   File "/usr/local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 222, in request
api_1           |     raise ApiException(http_resp=r)
api_1           | kubernetes.client.rest.ApiException: (403)

faced when try to parse k8s config body

Added boto3 to setup.py requirements Added eks engine Added eks kubeconfig template

ekhomyakova · 2018-06-06T15:46:15Z

+    'ACTIVE': config.get('CLUSTER_OK_STATE'),
+    'DELETING': config.get('CLUSTER_DEPROVISIONING_STATE'),
+    'FAILED': config.get('CLUSTER_ERROR_STATE'),
+    'UPDATED': config.get('CLUSTER_UPDATING_STATE')


'UPDATING' ?

@ekhomyakova I suppose those names comes from EKS, but I don't see updating in the list at all, so may be we can just remove updated from the list

for what i know the engine requires an updating state, so either we leave it or we need to put it to unknown

ekhomyakova · 2018-06-06T15:49:10Z

+                    'number': True
+                }
+            },
+            'roleArn': {


It's better to use snake_case everywhere

ekhomyakova · 2018-06-06T15:49:26Z

+                    'required': True
+                }
+            },
+            'subnetid': {


subnet_id

ekhomyakova · 2018-06-06T15:49:49Z

+                    'required': True
+                }
+            },
+            'securitygroupid': {


security_group_id or securitygroup_id

katyafervent · 2018-06-06T16:20:31Z

+        except Exception as e:
+            msg = 'Creating cluster {} failed with the following reason:'.format(self.cluster.id)
+            logger.exception(msg)
+            return False, msg


Please, return only "e" instead of msg, since message will be huge and looks like:

Error occurred with the following reason: Creating cluster failed with the following reason:'

katyafervent · 2018-06-06T16:21:49Z

+    def engine_status(cls, **kwargs):
+        try:
+            aws_access_key = kwargs.get('aws_access_key', '')
+            aws_secret_key = kwargs.get('aws_secret_key', '')


did you mean 'aws_secret_access_key'?

katyafervent · 2018-06-06T16:26:48Z

+    'ACTIVE': config.get('CLUSTER_OK_STATE'),
+    'DELETING': config.get('CLUSTER_DEPROVISIONING_STATE'),
+    'FAILED': config.get('CLUSTER_ERROR_STATE'),
+    'UPDATED': config.get('CLUSTER_UPDATING_STATE')


@ekhomyakova I suppose those names comes from EKS, but I don't see updating in the list at all, so may be we can just remove updated from the list

katyafervent · 2018-06-06T16:27:18Z

+        """
+        Implementation of :func:`~kqueen.engines.base.BaseEngine.provision`
+        """
+        # self.name = kwargs.get('name', 'noname')


katyafervent · 2018-06-06T16:28:06Z

+            return {}
+        state = STATE_MAP.get(response['cluster']['status'], config.get('CLUSTER_UNKNOWN_STATE'))
+
+        key = 'cluster-{}-{}'.format(response['cluster']['name'], response['cluster']['name'])


why names are used two times?

one of them should be the id, but EKS uses the name as id at the moment, the only other option is to use the ARN which is extremely long arn:aws:eks:us-west-2:012345678910:cluster/prod

katyafervent · 2018-06-06T16:29:34Z

+        try:
+            response = self.client.describe_cluster(self.cluster.metadata['id'])
+        except Exception as e:
+            msg = 'Fetching data from backend for cluster {} failed with the following reason:'.format(self.cluster.metadata['heat_cluster_id'])


self.cluster.metadata['id']

katyafervent · 2018-06-06T17:11:26Z

+            },
+            'subnetid': {
+                'type': 'text',
+                'label': 'Subnet Id',


Subnet/s Id - since more than one can be chosen. Ideally we need to get list of them and provide ability to pick subnets on the fly, but for now we need to parse string to the list at least (on the UI part)

katyafervent · 2018-06-06T17:54:59Z

+            aws_secret_access_key=aws_secret_key
+        )
+        try:
+            list(client.list_clusters())


can we used more lightweight request here? (since this request is sending for each cluster, even for list all clusters operation)

at least add maxResults parameter=1

or use another request maybe generate_presigned_url?

seems, No , coz afaik only list_clusters can provide valid check on all required User IAM roles/policies and etc.. I understand that its huge request, but we are locked by small api

- Template structure defined - Kubeconfig download fixed - Cluster states fixed - Engine status fixed - TODO added current issue related with kqueen-k8s-api methods: ``` api_1 | Traceback (most recent call last): api_1 | File "/code/kqueen/models.py", line 124, in status api_1 | 'addons': kubernetes.list_services(filter_addons=True), api_1 | File "/code/kqueen/kubeapi.py", line 238, in list_services api_1 | include_uninitialized=include_uninitialized api_1 | File "/usr/local/lib/python3.6/site-packages/kubernetes/client/apis/core_v1_api.py", line 14358, in list_service_for_all_namespaces api_1 | (data) = self.list_service_for_all_namespaces_with_http_info(**kwargs) api_1 | File "/usr/local/lib/python3.6/site-packages/kubernetes/client/apis/core_v1_api.py", line 14455, in list_service_for_all_namespaces_with_http_info api_1 | collection_formats=collection_formats) api_1 | File "/usr/local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 321, in call_api api_1 | _return_http_data_only, collection_formats, _preload_content, _request_timeout) api_1 | File "/usr/local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 155, in __call_api api_1 | _request_timeout=_request_timeout) api_1 | File "/usr/local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 342, in request api_1 | headers=headers) api_1 | File "/usr/local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 231, in GET api_1 | query_params=query_params) api_1 | File "/usr/local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 222, in request api_1 | raise ApiException(http_resp=r) api_1 | kubernetes.client.rest.ApiException: (403) ``` faced when try to parse k8s config body

Refactor EKS

- Typo fixes in EKS engine

Docs added

katyafervent

Please, add new pipfile.lock

katyafervent · 2018-06-13T12:55:50Z

+        },
+        'cluster': {
+            'node_count': {
+            # TODO unsupported, need to define template k8s input


I suggest to comment node_count then

katyafervent · 2018-06-13T12:57:31Z

+        - "-i"
+        - "<cluster-name>"
+        # - "-r"
+        # - "<role-arn>"


is it suppose to be commented?

for now yes

Heptio Auth configuration

naumvd95 · 2018-07-02T05:44:08Z

related issue kubernetes-client/python#514

naumvd95

lets say I'm ok with that, but in fact I just want to hide this PR from my notifications tab :)

naumvd95

lets say I'm ok with that, but in fact I just want to hide this PR from my notifications tab :)

Added eksengine to engines/__init__.py

f16ed10

Added boto3 to setup.py requirements Added eks engine Added eks kubeconfig template

mikezuniga requested review from katyafervent and naumvd95 June 6, 2018 15:40

ekhomyakova reviewed Jun 6, 2018

View reviewed changes

katyafervent reviewed Jun 6, 2018

View reviewed changes

naumvd95 and others added 4 commits June 10, 2018 11:35

Merge pull request #317 from Mirantis/eks-refactor

391f91c

Refactor EKS

Docs added

625c540

- Typo fixes in EKS engine

Merge pull request #318 from Mirantis/kqueen-eks

b016a1d

Docs added

naumvd95 requested review from ekhomyakova and katyafervent June 13, 2018 06:42

katyafervent reviewed Jun 13, 2018

View reviewed changes

naumvd95 and others added 2 commits June 20, 2018 15:40

Heptio Auth configuration

6144bcf

Merge pull request #321 from Mirantis/kqueen-eks

0c0df12

Heptio Auth configuration

naumvd95 added the TBD label Jul 2, 2018

naumvd95 approved these changes May 23, 2023

View reviewed changes

Conversation

mikezuniga commented Jun 6, 2018 • edited by naumvd95 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

katyafervent left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

naumvd95 commented Jul 2, 2018

Uh oh!

naumvd95 left a comment

Choose a reason for hiding this comment

Uh oh!

naumvd95 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mikezuniga commented Jun 6, 2018 •

edited by naumvd95

Loading