SKIL Documentation

Skymind Intelligence Layer

The community edition of the Skymind Intelligence Layer (SKIL) is free. It takes data science projects from prototype to production quickly and easily. SKIL bridges the gap between the Python ecosystem and the JVM with a cross-team platform for Data Scientists, Data Engineers, and DevOps/IT. It is an automation tool for machine-learning workflows that enables easy training on Spark-GPU clusters, experiment tracking, one-click deployment of trained models, model performance monitoring and more.

Get Started

Docker Image

Prerequisites

System Requirements

The minimum requirements for a single-node SKIL installation are:

  • Quad-Core Processor with AVX instructions
  • 16 GB RAM
  • Minimum 1Gbps network connection
  • 50GB of disk space

The following specs are recommended for production use:

  • Eight-Core or better Processor with AVX2 instructions
  • 64 GB RAM
  • Minimum 1Gbps network connection
  • 500GB+ of disk space
  • It is also recommended that the docker memory be set to a minimum of 8GB. Take a look here for more details.

Docker

To install Docker visit this link and select your platform.

Increasing Docker VM's Host Memory

If you are not using Linux you must increase the host VM memory. The way to modify the host VM can depends on the environment.

1. Legacy Docker Toolbox (Mac OS & Windows)

Using docker-machine and VBoxManage (Doesn't require VM deletion)

Open up a command line by pressing the start button and typing cmd.exe or opening Terminal.app on Mac.

docker-machine stop
VBoxManage modifyvm default --cpus 2
VBoxManage modifyvm default --memory 8192
docker-machine start

VBoxManage showvminfo default
docker-machine ssh default free

Using just docker-machine (Requires VM deletion)

docker-machine rm default
docker-machine create -d virtualbox --virtualbox-memory=8192 --virtualbox-cpu-count=2 --virtualbox-disk-size=50000 default

2. Docker (Windows)

Go to the system tray and right-click the docker icon. Then navigate to Settings > Advanced. See the images below for reference:

Right-click the docker icon in the system tray. Then nagivating to "Settings".

Right-click the docker icon in the system tray. Then nagivating to "Settings".

Modifying the compute resources.

Modifying the compute resources.

3. Docker (Mac OS)

Go to the menu bar and right-click the docker icon. Then navigate to Preferences > Advanced. See the images below for reference:

Docker preferences option.

Docker preferences option.

Changing the compute resources in the advanced tab.

Changing the compute resources in the advanced tab.

4. Docker (Linux)

If you're running native Docker instance over Linux (without any virtual machine), then you won't have to go through the steps for increasing the VM memory as the Docker runtime would have the whole machine for its use.

Pulling SKIL's Docker Image

To pull the SKIL image from the repo, use this command:

docker pull skymind/skil

This takes time and disk space

Since SKIL includes many frameworks the Docker pull can take up to 90 minutes depending on your Internet connection. If it stalls, please check your disk space. We recommend at least 15 GB of available space.

Run the SKIL server

To run a temporary SKIL server, run this command:

docker run --rm -it -p 9008:9008 -p 8080:8080 skymind/skil

Errors during run

If you get errors while executing the above command make sure the docker daemon is running. First verify you can run docker ps successfully. If not start docker from the start menu or Docker.app on Mac or using docker-machine start default and following the instructions from the command docker-machine env default

You may need to wait up to 5 minutes

SKIL will perform some installation steps on its first run and can take up to 10 minutes before it is ready to use.

On your local Linux machine or when using Docker-CE for Mac or Windows, you can access the SKIL UI by opening a browser window to http://localhost:9008 (or http://192.168.99.100:9008 instead if you are using Docker Toolbox. You can use the handy docker-machine ip command to find out this host address).

Make sure your Docker VM has at least 8 GB of RAM. (You can change it in the virtual box app.)

First Time Setup

For the first startup, you will need to do the two initialization steps below:

1. Obtaining SKIL License

For first time users, SKIL is going to ask you for license verification. To get that done, you will need to create a license for the community edition by visiting https://skymind.ai/activate. Fill up your details and after you receive your license details at the provided email address, paste it into SKIL and navigate to the next step.

2. Setting up a default Password

After setting up the license, you can setup a password for the user admin for future logins into the SKIL server.

If you already have a license file and want to persist data with it, see the Persisting Data

Persisting Data

To persist your notebooks and models servers when using SKIL you need to use Docker's data management features. See Managing Data in Containers for additional information.

The simplest way to persist your notebooks and model server configurations is to use persistent data volumes for SKIL. You can do this with the following commands:

docker volume create --name skil-data
docker volume create --name skil-conf
docker volume create --name skil-root
docker run -it --rm -v skil-root:/opt/skil -v skil-data:/var/skil -v skil-conf:/etc/skil -p 9008:9008 -p 8080:8080 -d skymind/skil

If you already have a license file with you, you can start the container in the following way:

docker volume create --name skil-data
docker volume create --name skil-root
docker run -it --rm -v skil-root:/opt/skil -v skil-data:/var/skil -v /path/to/license:/etc/skil/license.txt -p 9008:9008 -p 8080:8080 -d skymind/skil

Afterwards you can stop SKIL like so:

docker stop <container_name> # You can find it with "docker ps"

and start it back again with (depending on your configuration):

docker run -it --rm -v skil-root:/opt/skil -v skil-data:/var/skil -v skil-conf:/etc/skil -p 9008:9008 -p 8080:8080 -d skymind/skil

# OR

docker run -it --rm -v skil-root:/opt/skil -v skil-data:/var/skil -v /path/to/license:/etc/skil/license.txt -p 9008:9008 -p 8080:8080 -d skymind/skil

When using persistent data the SKIL installation steps it performs are cached and it will be ready to use within 2-5 minutes. While SKIL is starting, you'll sometimes see the following screen:

Docker Image


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.