MesosCon18 has ended
Welcome to MesosCon 2018 which will be held in The Village (969 Market St, San Francisco) between November 5th-7th, bringing together users and developers to share and learn about the project and its growing ecosystem.

Tickets are now available to purchase below.

Back To Schedule
Monday, November 5 • 1:00pm - 1:40pm
Scaling Mesos to thousands of frameworks

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

One of the strengths of Mesos is the ability to simultaneously run diverse schedulers. We recently had a use case that required running thousands of instances of different frameworks (Marathon, Cassandra, Spark, Jenkins) on a single cluster. While it is well understood that Mesos clusters can scale to tens of thousands of agents, running thousands of frameworks on them is still uncharted territory.

We set out to do scale tests to explore this territory. In this talk, we will present the treasures that we found and dragons encountered along the way.

We will describe the tooling we developed to monitor and execute the tests, the challenges we faced on the allocator such as offer starvation and fragmentation, the inevitable performance problems and surprising behaviours.

We will share our learnings, the things things we fixed, and some of the best practices that we recommend.

avatar for Gastón Kleiman

Gastón Kleiman

Staff Software Engineer, Mesosphere
Gastón Kleiman, Apache Mesos PMC/Committer, is a Staff Software Engineer at Mesosphere. He fell in love with distributed systems and infrastructure automation while contracting for Google, where he got to use Borg, MapReduce and other cool technology. That led him to work at Amazon... Read More →
avatar for Meng Zhu

Meng Zhu

Software Engineer, Mesosphere Inc.
Meng Zhu is an Apache Mesos committer and PMC member at Mesosphere, primarily works on resource allocation in Mesos. Previously, he received his PhD in Computer Engineering from University of Rochester, where he worked on operating system resource management.

Monday November 5, 2018 1:00pm - 1:40pm PST
  Breakout Session