MesosCon18 has ended
Welcome to MesosCon 2018 which will be held in The Village (969 Market St, San Francisco) between November 5th-7th, bringing together users and developers to share and learn about the project and its growing ecosystem.

Tickets are now available to purchase below.

Back To Schedule
Monday, November 5 • 3:30pm - 4:10pm
Kraken: p2p docker image distribution system

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Docker container is a foundational building block of Uber infrastructure, but distributing the docker images to all hosts in a system is a scaling problem we have been facing at Uber for some time.

We started with a single docker registry in 2015, gradually added replicas and cache layers. However as Uber grew larger and more complex, the number of hosts and containers also grew exponentially; Also as the deployment system become more automated, rebalance and rollback happens more frequently. Docker registry couldn’t catch up with the number of container being deployed, which grew 10x every year to millions of docker pull requests per day. Eventually we decided to take a P2P distribution approach to tackle this problem - the project is called Kraken.

Kraken is a P2P docker image distribution system. It’s loosely based on BitTorrent protocol, fully compatible with docker registry API, and supports pluggable storage backends like S3, HDFS, etc. It successfully solved scaling problems we saw under different scenarios, also greatly sped up container deployment.

This talk will cover:
- Docker registry scaling challenges
- Architecture of Kraken
- Design and optimizations decisions we took
- Production performance numbers

avatar for Yiran Wang

Yiran Wang

Software Engineer, Uber
Works on container and docker image related tools and services at Uber. TL of docker image builder Makisu, P2P docker registry Kraken, and Kubernetes migration.

Monday November 5, 2018 3:30pm - 4:10pm PST
  Breakout Session
  • Host Organization Uber