An Interest In:
Web News this Week
- April 26, 2024
- April 25, 2024
- April 24, 2024
- April 23, 2024
- April 22, 2024
- April 21, 2024
- April 20, 2024
Some of Our Sources
- The Logo Smith
- Web Designer Depot
- Naldz Graphics
- Wal You
- Specky Boy
- CSS Tricks
- Willems Lab
- Dev To
- Hashedout
- The Verge
Help Webnuz
Referal links:
March 16, 2013 12:35 am GMT
Airbnb just announced that it has open sourced Chronos, the company’s replacement for the standard time-based Cronschedule that is probably familiar to everybody who has ever used a Unix-based system. Chromos runs on top of Apache’s Mesos cluster manager and allows developers to execute shell scripts at set intervals and, unlike Cron, you can also trigger jobs by the completion of other jobs. Chronos, the team writes, was born out of Airbnb’s need to automate more of its data processing pipeline. Until last December, Airbnb was apparently using a single hourly Cron job that would start its complex processing pipeline. The problem with this, the company’s engineers write was that they “use a managed Hadoop product from Amazon, called Elastic Map/Reduce. High variance in network latency, virtualization and not having predictable I/O performance is an ongoing challenge in a cloud environment.” This increased the chance of something going wrong, so the team decided it needed a system that would allow for retries and would give its analysts the ability to see which jobs had failed. To make keeping track of Chronos even easier, the team also developed a web-based user interface that makes it easier to manage and run jobs than the usual text file Cron users have to rely on (Chronos also features an API, so you can always build your own GUI, too). Airbnb’s GitHub repository delves a little bit deeper into the details of how Chronos works and if you’ve got 45 minutes to burn and would love to hear more about Chronos, here is the tech talk the team did at Airbnb’s headquarters earlier this year:
Original Link: http://feedproxy.google.com/~r/Techcrunch/~3/jIxg5dhWrus/
Airbnb Open Sources Its Chronos Scheduler, A More Flexible Cron Replacement With A Web-Based GUI
Airbnb just announced that it has open sourced Chronos, the company’s replacement for the standard time-based Cronschedule that is probably familiar to everybody who has ever used a Unix-based system. Chromos runs on top of Apache’s Mesos cluster manager and allows developers to execute shell scripts at set intervals and, unlike Cron, you can also trigger jobs by the completion of other jobs. Chronos, the team writes, was born out of Airbnb’s need to automate more of its data processing pipeline. Until last December, Airbnb was apparently using a single hourly Cron job that would start its complex processing pipeline. The problem with this, the company’s engineers write was that they “use a managed Hadoop product from Amazon, called Elastic Map/Reduce. High variance in network latency, virtualization and not having predictable I/O performance is an ongoing challenge in a cloud environment.” This increased the chance of something going wrong, so the team decided it needed a system that would allow for retries and would give its analysts the ability to see which jobs had failed. To make keeping track of Chronos even easier, the team also developed a web-based user interface that makes it easier to manage and run jobs than the usual text file Cron users have to rely on (Chronos also features an API, so you can always build your own GUI, too). Airbnb’s GitHub repository delves a little bit deeper into the details of how Chronos works and if you’ve got 45 minutes to burn and would love to hear more about Chronos, here is the tech talk the team did at Airbnb’s headquarters earlier this year:Original Link: http://feedproxy.google.com/~r/Techcrunch/~3/jIxg5dhWrus/
Share this article:
Tweet
View Full Article
Techcrunch
TechCrunch is a leading technology blog, dedicated to obsessively profiling startups, reviewing new Internet products, and breaking tech news.More About this Source Visit Techcrunch