SKIL Documentation

Skymind Intelligence Layer

The community edition of the Skymind Intelligence Layer (SKIL) is free. It takes data science projects from prototype to production quickly and easily. SKIL bridges the gap between the Python ecosystem and the JVM with a cross-team platform for Data Scientists, Data Engineers, and DevOps/IT. It is an automation tool for machine-learning workflows that enables easy training on Spark-GPU clusters, experiment tracking, one-click deployment of trained models, model performance monitoring and more.

Get Started

You can integrate and access logs with SKIL in several ways. Logging configuration is useful for more complex devops and deployments where you need to obtain logs in an organized fashion from a large cluster.

Logback.xml Appenders

If your favorite logging service uses Logback appenders, you can place a custom logback.xml file in /etc/skil/logback.xml.

Do not remove existing appenders

If your SKIL installation has already placed a logback.xml file on your machine, do not delete any existing appenders in the file. Otherwise, you may not be able to retrieve logs from the HTTP API or other services. Instead, add appenders to your file.

Logstash

If you're using a tool such as Logstash, your file will look like:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <appender name="stash" class="net.logstash.logback.appender.LogstashSocketAppender">
    <host>MyAwesomeSyslogServer</host>
    <!-- port is optional (default value shown) -->
    <port>514</port>
  </appender>
  <root level="all">
    <appender-ref ref="stash" />
  </root>
</configuration>

JDBC Appender

If you prefer to use JDBC to store your logs, your logback.xml file will look like:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <appender name="DB" class="ch.qos.logback.classic.db.DBAppender">
    <connectionSource class="ch.qos.logback.core.db.DriverManagerConnectionSource">
      <driverClass>com.mysql.jdbc.Driver</driverClass>
      <url>jdbc:mysql://host_name:3306/datebase_name</url>
      <user>username</user>
      <password>password</password>
    </connectionSource>
  </appender>
  
  <root level="DEBUG" >
    <appender-ref ref="DB" />
  </root>
</configuration>

HTTP /logs API

You can also access logs by sending a POST request to the SKIL /logs endpoint. Your request body must contain the following data in application/json type:

{ "offset": 0, "rowCount": 100, "head": false }

The above will retrieve the 100 most recent log lines. See the /logs API reference for more details.

Logging