solvcon.cloud is seriously outdated

[yungyuc]

I am building infrastructure on GCE/GKE, and it doesn't seem to be necessary to continue work with AWS at this point. Since solvcon.cloud is seriously outdated, it may be deprecated.

For example, it still hardcodes bitbucket location for cloning source code. To bring it up to date may be like to redevelop the GCE script (contrib/gce/scg) for AWS.

We may discuss in this ticket how to proceed for developing GCE/GKE deployment and development environment.

[yungyuc]

@tai271828 I'd like to learn how do you think about the "cloud work". GCE/GKE looks like the way to go. The data center is geologically close to us and has good (low) latency. The price is reasonable. It has leading container cluster support (kubernetes). We may also test singularity.

How do you think we should proceed?

[tai271828]

I have not much idea about singularity yet[1]. However k8s is definitely a sufficient reason for us to evaluate the pros/cons to develop with GCE/GKE.

The first thing came to my mind when I read the introduction of singularity, is to use it to build a mixed HPC cloud. For example, using k8s to manage the singularity-based SOLVCON environment containers and run the real computing workload on real and physical nodes.

This is a very preliminary idea. I needs more comments and information. Comments are welcome.

[yungyuc]

An immediate use of GKE is for gitlab container-based CI. But before we have enough experience with running SOLVCON in container, it's still far out.

I suggest to pick up the docker project you executed a while ago and see how it may make building and running SOLVCON easier than using GCE with the new sgc script. Hopefully when we meet on Friday you have something to show.

Singularity hasn't yet entered the mainstream and we may wait.

[tai271828]

What if the goal could be "easier setup and run simulation flow, which less than 5 or less commands"?

This goal could be also divided into several phases as the following:

1. A local wrapper
2. A kvm-based and docker-based Beowulf cluster hosted locally
3. Similar to the above, but the containers are hosted on the cloud
4. Similar to the above, but SOLVCON is built on bare metals.
5. Capstone of the above
6. (Good to have) Playing SOLVCON with a DGX-1
7. (Good to have) Similar to the above, but SOLVCON is built and run on a mixed cloud

The CI stuff could be implemented possibly in each phase. This needs more investigation.

This blueprint summarizes only the very preliminary ideas. More details would come out after implementing prototype. Comments are welcome, thanks.

Local Wrapper

This phase is to build a wrapper to make the build/installation/run be easier.

I think to build and run SOLVCON is simple and easy now, but we may make it more simpler to academic people, who may or may not be familiar with how to use Unix-like style command lines.

The measurement of this phase: a flow with less than or equal to 5 commands including to install git etc.

Our 2017 bootcamp provides a very good demonstration of the usual flow to use SOLVCON. It is a good start. We could integrate the steps.

Capstone

The capstone could be comparison of the performance of each phase infrastructure.

DGX-1 Infrastructure

To build and run SOLVCON on DGX may need a lot of CUDA knowledge.

Mixed Cloud

An intuitive mixed cloud could be: the local nodes hosting driving scripts and managing a "computing load balance" to create/use/destroy the ephemeral nodes with high computing power if necessary.

[yungyuc]

I think @tai271828 your comments are also in favor of containerization. Being a popular solution for deployment in the SOA, containerization has some traits that's useful to HPC applications. I agree with you.

Could you also comment on how we should do with this issue?

[tai271828]

My quick preliminary survey shows the following ideas:

kvm-based Beowulf cluster

The library libvirt is popular and available in many distributions. Python libvirt library works like a charm, e.g. my prototype. I am going to implement this SOA solution.

docker-based Beowulf cluster

k8s is absolutely the solution for

• It could be hosted locally.
• It is popular and active open source project. We could get supports very well if necessary.
• It is compatible with GCE so we could migrate it to the cloud of the next phase easier.

Bare Metals

The first thing comes to me mind is still Canonical's MaaS/Juju solution. Openstack is also another candidate. This needs more investigation.

[Updates] OpenHPC may be another candidate as well.

DGX-1

Not having much idea yet. I have to get familiar its spec and then could be able to how to leverage it.

Mixed Cloud

TBD. The first thing comes to my mind would be a bare-metal master with VM or container-based services to manage local bare metal nodes for storage and performance computing nodes on the cloud.

Misc

According to our further discussion (see the thread below), we will remove the part of AWS along with the maturity of this plan.

[yungyuc]

Sorry for delay. Too much to catch up after a vacation :-)

Your prototype with libvert looks the most interesting. I think it's a good idea to keep on that path.

As to the subject of this issue, what should we do with solvcon.cloud? It's outdated and I doubt it works at all. If we want to move toward libvert, do we still need boto? And its dependencies? These questions should be answered.

[tai271828]

There will be 1st version of libvert prototype within few days. I am going to make it.

About solvcon.cloud, I don't think we have resource to continue maintaining boto stuff. I would say let's lower boto priority because we want that SOLVCON is easy to setup and use for academic people, who may not have accounts to use AWS service. By contrast, they are very likely to have a Google account, and it is easier for them to get one or more bare metals.

In conclusion,

• Focus on local and GCE/GKE stuff first.
• Lower boto related service maintenance.

What do you think?

[yungyuc]

It's exciting to know libvert will be working in days. Look forward to it. And sure, if you also think we don't need AWS, let's get rid of it. Do you mind to work on removing the module and its dependency from the code base?

[tai271828]

Yes, sure. I will remove AWS part of the code. ( It is still to make me sad to remove lovely code. )