Using the Digital Ocean API to Manage Cloud Instances

What's Digital Ocean?

Digital Ocean is one of the fastest growing web hosts, in part due to its fast SSD-based servers and inexpensive $5 monthly hosting plans. Spinning up instances on Digital Ocean for testing or long-term use is fast, easy and affordable. 

In Building an App Image to Resell at Digital Ocean, I walked you through how to launch a WordPress (or other app) instance and repeatedly clone it for clients. In this tutorial, I'm going to guide you in using the Digital Ocean API to programmatically manage our instances, also known as droplets, and to automate common tasks. I'll also provide a sample codebase for running very basic API operations written in the Yii Framework for PHP; you can get the code here on Github.

The Digital Ocean API

The Digital Ocean API allows you to manage Droplets and resources in a simple, programmatic way using HTTP requests. All of the functionality that you are familiar with in the Digital Ocean control panel is also available through the API, allowing you to script the complex actions that your situation requires.

For this tutorial, we'll integrate developer Antoine Corcy's Digital Ocean V2 PHP API Library into a Yii-based console application. You don't need a knowledge of Yii to use the application but you can learn more about it here: Introduction to the Yii Framework.

The Digital Ocean API authenticates your account via OAuth and is composed of close to a dozen high level areas:

• 
  Accounts: provides basic information about your Digital Ocean account.


• 
  Actions: a historical log of actions on the droplets in your account.


• 
  Domains & Domain Records: allows you to manage the domains used on droplets in your account as well as the DNS records associated with them.


• 
  Droplets & Droplet Actions: allows you to create, shutdown, restart, snapshot and more for your droplets.


• 
  Images & Image Actions: allows you to enumerate and manage the snapshot images you've taken of droplets on your account. 


• 
  SSH Keys: allows you to register an SSH key to install when creating a droplet so that password security is not required.


• 
  Regions: enumerates the geographical regions that Digital Ocean droplets can be created within.


• 
  Sizes: enumerates the droplet sizes that you can use when creating droplets.

Now that you know a bit about the API, let's dive in with our own application.

Setting Up Access Keys

In order to use the API, you need to activate Personal Access Tokens for your account. Log in to your account and visit the settings application page: https://cloud.digitalocean.com/settings/applications. Click Generate New Token, as shown below:

Make note of your access token below—Digital Ocean will only show it to you once:

Now, let's move on to our sample console application.

Our Console Application

In this tutorial, we're going to explore a small console application I've built that performs a few tasks related to managing droplets. You can install the application yourself from the Tuts+ GitHub repository and customize or extend it to implement additional API features that you desire. I've posted a detailed installation guide for the console application at my website. You can also explore my generic installation guide for Digital Ocean apps.

Configuring API Access

Again, we're using Antoine Corcy's Digital Ocean V2 PHP API Library to access the API. 

We've built a component called Ocean.php which acts as a model for using his library. It's at /app/protected/components/Ocean.php.

Yii loads your access token from the Ocean.ini file, described in the Digital Ocean Console Application Installation Guide, and instantiates a digitalOcean API object.

<?php

use DigitalOceanV2\Adapter\BuzzAdapter;
use DigitalOceanV2\DigitalOceanV2;

class Ocean extends CComponent
{
  private $adapter;
  private $digitalOcean;
  
   function __construct() {     
     // create an adapter with your access token which can be
     // generated at https://cloud.digitalocean.com/settings/applications
     $this->adapter = new BuzzAdapter(Yii::app()->params['ocean']['access_key']);
     // create a digital ocean object with the previous adapter
     $this->digitalOcean = new DigitalOceanV2($this->adapter);
  }

Fetching Droplets

Now, let's fetch a list of our active droplets. In /app/protected/models/Droplet.php, our sync method invokes the Ocean components and gets the droplets:

    public function sync() {
    $ocean = new Ocean();
    $droplets = $ocean->getDroplets();
    foreach ($droplets as $d) {
      $droplet_id = $this->add($d);
    }      
  }

Here's what the Ocean getDroplets method looks like: 

  public function getDroplets() {
    // return the action api
    $action  = $this->digitalOcean->droplet();
    // return a collection of Action entity
    $actions = $action->getAll();    
    return $actions;
  }

Note: the basic console application just does a one-way download synchronization of our droplet listings. You could implement more features on your own, including removing droplets that have been deleted in the cloud.

Here's our Droplet model's add function. If the droplet already exists, we just update the record:

public function add($droplet) {
     $d = Droplet::model()->findByAttributes(array('droplet_id'=>$droplet->id));
    if (empty($d)) {
      $d = new Droplet;
    }
    $d->user_id = Yii::app()->user->id;
      $d->droplet_id = $droplet->id;
      $d->name = $droplet->name;
      $d->vcpus = $droplet->vcpus;
      $d->memory = $droplet->memory;
      $d->disk = $droplet->disk;
      $d->status = $droplet->status;
      $d->active =1;
     $d->created_at = $d->created_at;
     $d->modified_at =new CDbExpression('NOW()');          
     $d->save();
    return $d->id;
   }

If you wish to extend the model's features, Digital Ocean offers a wide variety of Droplet API actions and Corcy has a list of clear examples here.

Fetching Images

Next, we'll use the API to fetch a list of our current images. Images are snapshots, essentially backups, taken of a server instance at a given point in time.

Our Snapshot.php model has a sync operation which requests a list of your images and adds them individually to the database:

   public function sync() {
     $ocean = new Ocean();
     $snapshots = $ocean->getSnapshots();
     foreach ($snapshots as $i) {
       $image_id = $this->add($i);
       if ($image_id!==false) {
         echo $image_id;lb();
         pp($i);        
       }
     }          
   }

Here's the Ocean component getSnapshots code:

  public function getSnapshots() {
    // return the action api
    $action  = $this->digitalOcean->image();
    // return a collection of Action entity
    $actions = $action->getAll();    
    return $actions;
  }

Here's the Snapshot model add code—we ignore Digital Ocean's stock application images which are distinguished as public:

   public function add($snapshot) {
      $i = Snapshot::model()->findByAttributes(array('image_id'=>$snapshot->id));
     if (empty($i)) {
       $i = new Snapshot;
       $i->created_at =new CDbExpression('NOW()');          
     }
     if (isset($snapshot->public) and $snapshot->public ==1) {
       return false; // no need to save public images right now
     } else 
       $i->user_id = Yii::app()->user->id;       
       $i->image_id = $snapshot->id;
       $i->name = $snapshot->name;
       $i->region = $snapshot->regions[0];
       $i->active =1;
       $i->modified_at =new CDbExpression('NOW()');          
      $i->save();
    return $i->id;
    }

Automating Snapshots

As we discussed in Building an App Image to Resell at Digital Ocean, it's useful to automate the creation of snapshots which you can transfer to clients and customers. Unfortunately, there is currently no way to clone or transfer an image by reference; each time you transfer a snapshot to another account, it's gone.

Because Digital Ocean requires you instantiate an Image as a Droplet and power it off before taking a new snapshot, creating snapshots repetitively is a time-consuming manual process. It doesn't help that Digital Ocean powers on droplets again after taking snapshots—this just slows the process down.

Since the API doesn't accept requests while other operations are pending, we have to build a table to track background actions and use a cron job to repeat the power off, snapshot operation. Here's how it works:

Visit the Images page and click View for the snapshot you would like to clone. Then, click the Replicate menu option to the right. 

This will create a droplet and add an entry to the background Action table with this image_id and droplet_id. The end_stage is a constant you can set indicating the number of duplicates to create.

Here is the Snapshot model's replicate() method:

    public function replicate($id) {
        // look up image_id
      $snapshot = Snapshot::model()->findByAttributes(array('id'=>$id));
    // create the droplet 
      $ocean = new Ocean();
      $droplet_id = $ocean->launch_droplet($snapshot->name,$snapshot->region,$snapshot->image_id);
    // add command to action table with droplet_id and image_id
      $a = new Action();
      $a->droplet_id = $droplet_id;
      $a->snapshot_id = $snapshot->image_id;
      $a->action = Action::ACTION_SNAPSHOT;
      $a->status = Action::STATUS_ACTIVE;
      $a->stage = 0;
    // user settable constant for number of replications to make
      $a->end_stage = Snapshot::NUMBER_REPLICATIONS;
      $a->last_checked = 0;
      $a->modified_at =new CDbExpression('NOW()');          
      $a->created_at =new CDbExpression('NOW()');          
      $a->save(); 
    }

The cron task will ping https://ocean.yourdomain.com/daemon/index to regularly process the action table. Any unfinished overdue items will request another snapshot.

Here is the Action model's process() method:

    public function process() {
  set_time_limit(0);
  // look for overdue actions  
  $todo = Action::model()->overdue()->findAllByAttributes(array('status'=>self::STATUS_ACTIVE));
  foreach ($todo as $item) {
    if ($item->action == self::ACTION_SNAPSHOT) {
          $result = Snapshot::model()->take($item->id);          
    }
  }
}

The snapshot process will shut down the droplet, pause 20 seconds to wait for the droplet to shutdown, and request a snapshot.

Here is the Snapshot model's take() method:

 public function take($action_id) {
    $result = false;
      $a = Action::model()->findByPk($action_id);
    $snapshot = Snapshot::model()->findByAttributes(array('image_id'=>$a->snapshot_id));
    $ocean = new Ocean();
    // attempt shutdown
    // take snapshot
    $result = $ocean->snapshot($a->stage,$a->droplet_id,$snapshot->name,$snapshot->region,$snapshot->image_id);
    // if snapshot was successful
    if ($result) {
      // increment stage
      $a->stage+=1;
      // if last snapshot replication complete, end action
      if ($a->stage >= $a->end_stage)  
        $a->status = Action::STATUS_COMPLETE;
    } 
  // either way, update last_checked
    $a->last_checked = time();
    $a->save();              
  return $result;
  }

Here's the code in the Ocean component for actually making the API calls:

public function snapshot($stage,$droplet_id,$name,$region,$image_id,$begin=1,$count=3,$size='512mb') {
    $no_sleep = false;
    $name = str_replace("_","-",$name);
    $droplet  = $this->digitalOcean->droplet();
      try {
        echo 'Shutting down '.$droplet_id;lb();
        $shutdown = $droplet->shutdown($droplet_id);
      } catch (Exception $e) {
          $err = $e->getMessage();
          echo 'Caught exception: ',  $e->getMessage(), "\n";
          if (stristr ( $err , 'already powered off')===false)
            return false;
          else
            $no_sleep = true;
      }
    if (!$no_sleep) {
      echo 'Sleep 20 seconds for power off...';lb();
      sleep(20);              
    }
    echo 'Take snapshot of '.$droplet_id.' named '.$name.'-copy-'.$stage;lb();
    try {
      $snapshot = $droplet->snapshot($droplet_id, $name.'-copy-'.$stage); 
    } catch (Exception $e) {
        echo 'Caught exception: ',  $e->getMessage(), "\n";
        return false;
    }
    // shutdown and snapshot successful
    return true;
  }

If you visit the Digital Ocean website to view the droplet, you'll see the action in progress:

If the snapshot is successful, it returns to the Snapshot model to increment the stage. When the number of stage replications has finished, the action is completed.

You can visit the Images page at the Digital Ocean website to see your replicated snapshots:

After the images are created you can delete the droplet manually—or you can extend the code to do so when STATUS_COMPLETE is reached. If you do not delete the droplet, you will be charged for it.

Note that at this time, the API does not offer the ability to transfer a snapshot to an email address, so you will need to continue to do this manually through the web interface.

What's Next?

I hope that you've enjoyed this tutorial and find Digital Ocean to be a useful service in your portfolio of tools and hosting providers. In the next tutorial we'll explore the Digital Ocean DNS service.

Please feel free to post your questions and comments below. You can also reach me on Twitter @reifman or email me directly. Follow my Tuts+ instructor page to see future articles in this series.

Related Links

• The Digital Ocean API


• Digital Ocean V2 PHP API Library


• Digital Ocean Console Application Installation Guide