New python package use

#1

A new python package is released and I want to try its capabilities. What process should I undergo in order to use it in C3?

The previous experiment with the new package is successful and I want to have it available to deploy models. Which steps should I go through?

0 Likes

#2

Assuming you are using jupyter as a service (something like your-env-name.c3iot.com/jupyter)

You have two options:

  1. Experimentation only:
from c3notebook import c3_utils
c3_utils.refresh_kernels(local_only=True)
  • Refresh the page so the kernel appears among the options
  1. Production:
  • Create an ActionRuntime in the package.json of your package in your sources (note that in 7.9 it will become its own type, and the definition moves to seed data).
{
  "name": "myPackage",
  "description": "...",
  "author": "You",
  "dependencies": ["someOtherPackage"],
  "runtimes": {
    "py-myRuntime": {
      "language": "Python",
      "runtime": "CPython",
      "connector": "remote-types",
      "runtimeVersion": "3.6",
      "repositories": [
        "https://repo.continuum.io/pkgs/main"
      ],
      "modules": {
        "conda.numpy": "=1.14.2",
        "conda.scikit-learn": "=0.19.0",
        "conda.scipy": "=0.19"
        }
      }
    }
}
  • Provision your package.

  • Open a new notebook and run (and let it finish):

from c3notebook import c3_utils
c3_utils.refresh_kernels(kernels=["py-myRuntime"])
  • Refresh the page so the new kernel appears among the options
1 Like

#3

Hi,

This is very interesting. Can I install packages also via pip with the needed changes in package.json?

Perhaps by pointing repositories to my private pip server and replacing prefix conda by pip in modules list ?

Thanks
Fernando

0 Likes

#4

@fernando.gargiulo I have never tried to install packages in an ActionRuntime from a private pip server. I guess this will need the C3 server hosts to have the credentials to access your pip server. As far as I know, passing these credentials to the server is not handled natively in 7.8 from package.json for now, but I could be wrong. If your private pip server can be accessed without credentials, give it a try !

Concerning pip vs conda install, you are right, you can absolutely make:

{
  "name": "myPackage",
  "description": "...",
  "author": "You",
  "dependencies": ["someOtherPackage"],
  "runtimes": {
    "py-myRuntime": {
      "language": "Python",
      "runtime": "CPython",
      "connector": "remote-types",
      "runtimeVersion": "3.6",
      "repositories": [
        "https://repo.continuum.io/pkgs/main"
      ],
      "modules": {
        "conda.numpy": "=1.14.2",
        "conda.scikit-learn": "=0.19.0",
        "conda.scipy": "=0.19",
        "conda.pip": "=19.0.3",
        "pip.folium": "==0.6.0"
        }
      }
    }
}

Note that I added “conda.pip”: “=19.0.3” to install pip and then note the double “=” for “pip.folium”: “==0.6.0”.

I hope this helps,
Nathan

0 Likes