Fargate: Like Stargate but somehow less good

AWS Fargate is a great service. As of this writing recent adjustments in how AWS handles reservations for compute give even the budget conscious more incentives to use it. There are however some major limitations that might or might not impact you depending on your needs.

Some of them are in my opinion major issues when it comes to moving legacy applications to containers, some are simple deal breakers for some classes of software and others can simply be ignored.

In this post I'll lay out parts of my beef with the service, because why not. Also someone might actually find this useful or heaven forbid entertaining.

No persistent storage

Now this is the big one, this is the deal breaker for quite some older applications. If your application keeps state and it does so on a disk Fargate has nothing in place to deal with your persistence mechanism.

But, but EFS I hear people screaming from the sidelines. Yes you can mount an EFS into your Fargate machines. This is technically true you can, given that you know what you are doing, install the packages you need to mount an NFS on your machine, provision some storage in EFS, mount that storage into one or more containers and enjoy all the wonders of network file systems that we know and love.

I have been there I have tried this. The experience was as expected less than pleasant.

Lack of log drivers

For ECS you basically get the full range of log-drivers that are available to the Docker runtime. This provides you with a lot of flexibility in terms of integration with existing or third-party log analytic solutions.

Fargate is far more limited as it only offers splunk, awslogs and firelens. True splunk is very common, awslogs is not that expansive (if you only use a little of it) and not that terrible to use (again if you use it only a little), and you can build your own bird feeder after putting your stuff through firelens which is basically fluentd.

The fact remains, if you want to either keep loads of logs, do any form of serious enrichment or keep them in a not afore mentioned solution like ChaosSearch or ELK/EFK you will have to shell out either in time, money or both.

No daemonized services

Let us assume that you need to have some form of functionality across your entire fleet of services. Just for fun let us say something like system call monitoring for intrusion detection.

Over in Kubernetes land you would use the DaemonSet pattern for this. That is a container(pod) that you are guaranteed will run on every physical(virtual) machine that is part of your cluster.

ECS has a very similar feature though they call it SchedulerStrategy: Daemon. This however will not work with Fargate for somewhat obvious reasons.

This means you either use a sidecar approach or run more than one service per container. While the first is a well understood approach and the preferred way to solve this type of problem it comes with its own set of drawbacks.

It is:

• more expensive than DaemonSet
• more effort to maintain
• does not work for every use case

see our example.

This might leave you stuck with running more than one service per container, a thing they are not designed for. Yes it is possible, but it is not ideal and it opens a whole new can of worms.

No container console access

Yes, yes, yes you should not have console access to containers because that will make you treat them like pets not cattle.

Let's just say that no console access and quick, efficient troubleshooting do not go together. At lease in my experience.

That being said console access requires quite a bit of discipline, especially if you are new to the world of cattle, to not become a quagmire real fast.

You might want to take some time off from the console and accept the risks of that if you are just now entering the container game.

Container insight is hard

Now what do I mean by that? Ask any engineer worth their salt and they will tell you that unless you have the experimental data to test your hypotheses the best you can do is guess and guesses often does not hold water.

Fargate makes it very hard to get at some very basic metrics. CPU and memory usage, network statistics and characteristics, disk I/O, all things you want to know for various reasons.

Operationally they give you the ability to determine causes of issues or at least point you in the right direction. They also enable capacity planning and cost optimization. From a security perspective they serve as inputs to a team with advanced anomaly detection capabilities and from a software engineers perspective they hint at where you might want to optimize your application.

As I said, Fargate does not offer these metrics so either roll your own collection mechanism or ignore them at your peril.

Linux Capabilities are limited

Depending on your requirements this might also be a deal breaker for you. If you do not know what Linux capabilities are the gist of it is that they are flags that allow a process, like a container to do things usually only root can do.

An example would be IPC_LOCK which is required if you need to prevent memory swapping. This might be required for processes that hold very sensitive information in memory which you do not want to be written to disk during runtime. Supported capabilities can be found here (opens new window). Be aware of the NOTE section which actually stating what Fargate can and can not do.

Conclusion

When all is said and done the bottom line is this. I feel a lot of pain when using Fargate for a huge variety of reasons, some of them outlined above. Still if it fits your bill and use case you should absolutely use this service. It is just less complex then dealing with the issues of multiple layers of auto-scaling and workload scheduling you will eventually run into when backing your ECS with EC2 (and want to do it right).