Request: Task networking that gives task specific IP address without a dedicated ENI

[svella]

I was really excited about task networking until I realized the documented limitation:

Each task that uses the awsvpc network mode receives its own elastic network interface, which is attached to the container instance that hosts it. EC2 instances have a limit to the number of elastic network interfaces that can be attached to them, and the primary network interface counts as one. For example, a c4.large instance may have up to three elastic network interfaces attached to it. The primary network adapter for the instance counts as one, so you can attach two more elastic network interfaces to the instance. Because each awsvpc task requires an elastic network interface, you can only run two such tasks on this instance type. For more information about how many elastic network interfaces are supported per instance type, see IP Addresses Per Network Interface Per Instance Type in the Amazon EC2 User Guide for Linux Instances.

Unfortunately this means that because of the ENI per instance limitations that task networking is only useful for running largish tasks that claim a significant portion of the EC2 instance's resources. For instance, I could easily host 15 instances of my application (each representing a single tenant) on a t2.xlarge, but can't use task networking because a t2.xlarge only supports 3 ENI's.

I'm guessing the primary reason for this limitation is because security groups are assigned to ENI's rather than to IP addresses. If this is the case, how about an option that would allow multiple tasks using the same security group to share an ENI but still get a unique IP address?

[dls314]

Each task that uses the awsvpc network mode receives its own elastic network interface

This seems to be worse than necessary to the point where I wonder if it's correct?

By comparison, the amazon-vpc-cni-k8s plugin manages to use multiple addresses per network interface, so you can at least get 19 kubernetes pods on an m4.large

Here, if I read it correctly, you could only have 1 ecs task using awsvpc network mode on an m4.large which seems wrong; I would expect the same 1 task per IP address per interface.

[svella]

dls314

This seems to be worse than necessary to the point where I wonder if it's correct?

I also found it hard to believe at first, but it comes straight from the documentation and I'm pretty sure it's true. The reasoning behind it (as I understand it) is that Security Groups are assigned to network interfaces, not to IP addresses, and allowing Security Groups to be assigned to individual tasks was one of the design goals of the awsvpc networking feature.

[dls314]

Ah, I'd missed the reasoning you mentioned. I guess I'm still waiting for EKS :-)

Service Discovery was a compelling reason for awsvpc network mode and brought me here to check my understanding, but the density hit for the ENI limits is a 🤷‍♂️

[ghost]

any update on this issue????

[ofiliz]

The VPC abstraction for a network interface is ENI. ECS uses native VPC abstractions to give tasks full functionality on par with EC2 instances.

We think the proper solution to this problem is to increase ENI densities. We are tracking this effort in our public roadmap:
aws/containers-roadmap#7

We'd love to hear your feedback in our roadmap repo! I'm going to close this issue as it is functionally a duplicate of the roadmap item.