Add limit (maximum simultaneous commands) and throttle (delay between commands) to Phage
Summary:
Ref T12218. Ref T2794. This just provides a couple of general purpose tools to slow Phage down a bit if the speed is getting TOO EXTREME.
In particular, secure updates from itself so if all 4 nodes get hit simultaneously the deploy doesn't work. It deploys cleanly with --limit 1.
A separate change to the phage remote workflow in Instances exposes these options as flags:
- --limit: Run only this many commands simultaneously.
- --throttle: Wait this long between starting commands (for example, --throttle 0.25 will only start 4 commands per second).
Test Plan:
- Ran phage remote status on secure with various --throttle and --limit arguments, and without any arguments.
- Ran phage remote upgrade --limit 1 on secure, got a clean upgrade.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12218, T2794
Differential Revision: https://secure.phabricator.com/D17396