Posts Tagged ‘docker’

End to end testing in kubernetes

December 7, 2015

Something that I continue to be interested in is the possibility of end to end testing using kubernetes. The project has come a long way since its launch on github a little over a year ago, and it is interesting to see where it might head next. The purpose of this post is to explore the focus of my own particular fascination with it – the possibility of end to end testing complex workflows built of multiple microservices.

Currently kubernetes does have an end to end test suite, but the purpose of this seems largely to be for self service of the kubernetes project. However, there are the seeds of something with broader utility in the innards of the beast. Some contenders downstream have started building ways to render the possibility of writing end to end tests for applications easier, such as fabric8-arquillian for openshift applications. More recently, about twenty days ago, there was a pull request merged that looks like it might go a step further towards providing a more user-friendly framework for writing such tests. And the documentation for the system is starting to look promising in this regard, although a section marked examples therein is still classified as ‘todo’.

But why this obsession with end to end testing this way? Why is this so important? Why not simply as a tester write browser tests for an application?

The problem is that browser tests do not capture a lot of what is going on behind the scenes, but really only the surface layer, the user facing component of an application. To truly get more immediate feedback about whether or not a particular part of a workflow is broken, one needs something that can plug into all the backend machinery. And, in the world of the modern stack, that essentially means something that can plug into containers and trigger functionality / or monitor things in detail at will.

Kubernetes is of course not the only technology that has promise here. The more fundamental building block of Docker is starting to become quite versatile, particularly with the release of Docker 1.9, and the Docker ecosystem (Docker compose, swarm, and machine) may yet have a role to play as a worthy competitor, in the quest for the vaunted golden fleece of the end to end testing champion merino.  Indeed, anything that might well provide a practical and compelling way to end to end test an application built out of multiple containers is of interest to me, and although I am currently sitting on the kubernetes side of the fence, I could well be convinced to join the docker platform crowd.

My current thinking of a system that provides great end to end testing capability is the following: it should use docker (or rkt) containers as the most atomic components. One should have the ability to hook into these containers using a container level or orchestration service level API and activate various parts of the micro service therein which have been built in such a way as to facilitate a proper end to end test. There is however a problem here though, and it is due to the need for the security of the system to be watertight. So altering production code is understandably not particularly desirable here.

Instead, it would be great to somehow be able to hook into each docker container in the pathway of an application, and read the logs along the way (this can be done today). Also, however, it would be great to be able to do more. One idea might be to write code in the ‘test’ folder of the application next to the unit tests, that will only be run when the testing flag is set to true. Or, alternatively, maybe one could add api endpoints that are only opened in the staging environment, and are switched off in production (although this still of course may introduce unacceptable risks). If one had these additional endpoints, one could then read the state of particular variables as data flows through the application in the test, and check that everything is functioning as it should be.

The alternative to directly debugging code in this way of course is to simply log the value of a variable at a particular point, and then read the logs. Maybe this is, indeed, all that is required. Together of course with write tests for all of the standard endpoints of each container, and checking that they are working as they should be. There might be an elegant way of writing helper libraries to log something out to a docker log file for purposes of testing only, and then expose this information cleanly via some form of endpoint.

Anyway, the area of end to end testing multi container applications is quite an interesting one and I am keen to see how it develops and eventually matures. Together with monitoring tools like Prometheus, there is a large amount of potential for good work to be done in this direction and I am excited about where things might eventually lead with this.

Advertisements

Kubernetes, Docker, and Mesos

November 30, 2014

Recently I noticed a couple of interesting new developments emerging in the area of cloud deployment and distributed computing with virtual machines. A while ago, I experimented with dotcloud (in about 2011, 2012) when it was in its early start up stages, and found the tools therein a joy to use and play with setting up (and running) sites based on dotcloud’s dedicated servers. Later on, when dotcloud moved into their production phase, I let that particular venture slide, but found their release of Docker to be a further amazing development.

Docker in and of itself is very powerful, but does not afford the full flexibility and utility that dotcloud’s engine and tooling does, as their tooling allows / allowed one to assign a group of docker machines, with some form of ‘scaling’ factor, and with different groups of machines being assigned to particular microservices driving a web application. So, for instance, one might have a MongoDB database as a backend, and a java service for the frontend to implement a Spring application. Hence I was pleasantly surprised to discover, through reading the 5 year milestone blog entry for the release of the go programming language, that google has deployed a project called Kubernetes (from the Greek, meaning ‘helmsman of a ship’), also on github, that does exactly that – where one has the idea of a ‘pod’ of docker containers, with isolated volumes, and replication controllers, and microservices. All very exciting!

Both Docker and Kubernetes, it turns out, are built on Go, which is a programming language that has been designed and built / optimised specifically for cloud programming. Having discovered these new animals on github that seem to me likely to become systemically pivotal to the next steps in the web’s continued growth and evolution, I did some more reading on projects that have been implemented with Go and discovered the existence of Apache Mesos, which is essentially a way of abstracting computational resources from individual virtual machines, and is a step towards an operating system for distributed computing. Mesosphere on Github is a project consortium whose mission seems to be in line with said vision – and they have a project “Kubernetes-Mesos” that I think would be well worth watching, as well as “Marathon” which aims to be an “init” analogue for multiple VMs on a Mesos cluster, I think, may also become quite pivotal in the months and years ahead.

For a more mainstream take on these matters, there is this article at Wired that might be of interest for further reading on google’s kubernetes initiative. Here is a quote from the article in question:

“It’s part of an arms race. There are literally dozens of tools coming out,” says Solomon Hykes, the chief technology officer at Docker and the driving force behind the company’s software containers. “But Google joining that battle–with code that comes from their massive experience–helps show where this kind of thing will go.”

All quite fascinating.

But what does this mean for everyday folk or interested hobbyists like me or you?  Well, certainly, I would not mind deploying a (very small) Kubernetes cluster using rented cloud resources (eg, on GCE, Amazon EC2, Rackspace, Azure, etc) as part of a hobby project at some point.  The ability to scale is also enormously compelling, particularly for a small player who is interested in managing their OPEX to accommodate growth in their customer base.  More fully blown type implementations, such as incorporating Apache Mesos, are perhaps more suited to bigger players, say corporate type programming / systems engineering.  Of course, the number of corporations that would actually make use of the sort of tooling described in this article in the next one or two years could probably be counted on the fingers of one hand (after all, how many Twitters and Googles does the world have, today?  Not very many, I would wager).

However I am reminded of the apocryphal quotation (circa 1943) that there would maybe be a market for perhaps 5 computers in the world.  So maybe it will be the case with technological innovations such as this as well.  Maybe by 2030, we will see relatively widespread adoption of this sort of thing by large companies, even SME, or even individuals, routinely using 1000s of VMs, as a user of a computer today would not think twice of using billions of transitors today on a single machine.

Latest developments in the PaaS universe

June 28, 2013

Hi folks,

I’ve been considering the feasibility of maybe setting up a small PaaS on RightScale using Amazon EC2.  There are a few bits and pieces that might facilitate this.  However, most recently, I came across this interesting post, which pointed me to Dokku, a project that is only a few days old, but looks quite promising and very interesting.

An alternative strategy for PaaS cloud hosting might be to write a RightScript or three and get Sandbox installed subsequent to Docker on a virtual box on EC2, via this tutorial (a key difference being that RightScripts would likely be taking the place of Vagrant in setting up the system, if using RightScale).  However Docker itself only supports Python (as yet – of course, the project is still young!), and does not yet (it would seem) support databases, as required with Django apps.  Dokku, however, I believe supports Django and a number of other techs that are not even Python based – and it has some quite clever chaps from MIT, I believe, driving it.

Dokku builds not only on docker, but makes use of a couple of other useful github projects in order to work its magic.

Essentially it should allow the admin to deploy multiple data-driven websites / cloud services (eg, data monitoring / data feed input, then email / sms notify relevant subscribed users, say by google cloud messaging) / applications on the same server, but, according to the project description, it does not yet support multitenancy – which I assume would mean multiple admins / admin user logon (controlled by master admin), not does it yet support multihosts (ie, scaling to multiple virtual boxes, or failover between same).

More here.

Two excellent pieces of news

May 22, 2013

Hi folks,

I’m afraid this is not another content post, but the following items may be of interest.

The first is that Unity is now allowing developers who use their software to deploy not only to standalone, and to the Unity webplayer, but now also to Android and Apple mobile platforms for free.  In order to do this, merely open up Unity, open the “Unity” tab, click “Manage License” and then select “Check for Updates”.  Your license will be updated automatically, and mobile unlocked.  Via this post.

Another interesting development – you may recall that earlier I wrote about dotcloud’s amazing opensource release of Docker.  Now the same chaps have made it even easier to independently deploy “sandbox” style applications on same, via their release of a new project on their GitHub pageSandbox.  The original blog post describing this release is located here.

I have not yet investigated this wonderful new tool, but if you, the reader, are interested in checking it out, the installation instructions, both for docker and sandbox, can be found at this location.

dotCloud is going OpenSource

April 13, 2013

Hi folks,

I have now moved my blog to this location, since dotCloud’s Sandbox mode is now moving to an OpenSource implementation, which I have to say is quite commendable of the chaps.  Since I can’t currently afford to maintain my own servers on a 24/7 basis, I consider this an affordable low maintenance solution.

However this does lead me to become more motivated in order to actually build a live application on dotCloud that meets their desired criteria: “Our core competency is and will continue to be the operation and support of large-scale cloud services, for tens of millions of visitors, 24 hours a day, every day.”  Certainly something to aim for!  I’m not entirely certain the nature of a service that I would like to build, but, while I’m casually brainstorming:

  • it could be a service for building rooms for online collaboration, via manyuser-video conferencing, leveraging off TokBox; I would charge some nominal (small) fee for the service on an hourly basis, and then pay TokBox a portion of the take – since TokBox is fairly cheap (I think $200 / 250 for the first 75000 hours of usage per month total), this doesn’t seem like a poor strategy.
  • an online wiki.  Running on mediaWiki, I could host a not for profit specialist wiki focusing on some area that wikipedia might not focus on in the depth that I might like – eg a research wiki.
  • a forum registration service.  Running multiple user forums, the profit model would be advertising revenue and banner space.
  • a game.  I had difficulties with doing this before, however, in terms of where I wanted to take it.
  • an online provider of digital course content that I create, or
  • a provider for MOOCs (massive open online course) that simplifies the process of creating an online network of materials for students to use in their courses.

My impression is that the first and third points are probably the ones I’m most likely to want to try.  And the third would probably be the best to get started with.

The first step to enact the forum idea in particular might to consider Django as the technology to approach the matter.  I know that the lightbird tutorials cover forums in this respect, so I would not be starting from scratch.  It would then be simply a matter of extending / abstracting away from a single forum to creating a database of forums.

I guess in terms of how I would go about testing prior to implementation is that I would deploy on a Sandbox server, self-hosted, then contact dotCloud should I mean to move to Live / Enterprise mode.  Broadly speaking though, I think consideration of a live service is really only worthwhile if one has a passion or driving need to work in an area and sees an opportunity for improvement.  And largely speaking I’m learning a bit more fooling around with Unity currently! and see no obvious ways to simplify online transactional foolishness.  Learning is perhaps the most important thing for me at the moment – knowledge is, afterall, the best enabler of serendipity.