Resizing Jupyter session resources


#1

Hi,

I would like to change the amount of resources (memory and cores) of the container where my ipython kernel is running.

I found this thread which partially answers my question:

Now, two questions are left:

  1. How should I modify the snippet in order to change CPU or other kind of relevant configurations?

  2. Would I run in the same error as the one raised in the post?

Thanks
Fernando


#2

A normal user is only allowed to change the amount of memory available to his/her container. Other resources (instance type and disk space) are configured by System Administrators.

As a user to change the amount of memory you can access you can execute the following script:

function setJupyterSessionMemory(mem_in_mb) {
    var taskConfig = AwsEcsTaskDefinition.make({
        "id": "ecstd-jupyter"
    }).getConfig();
    var containerConfig = taskConfig.containerDefinitions;
    containerConfig[0].memoryReservation = containerConfig[0].memory = mem_in_mb;
 
    AwsEcsTaskDefinition.make({
        "id": "ecstd-jupyter"
    }).setConfigValue('containerDefinitions', containerConfig);
    console.log("Jupyter session set to " + mem_in_mb + " MB");
}
 
setJupyterSessionMemory(8*1024) // 8GB

Important: for the change to take effect you need to restart your container (don’t worry all your notebooks are saved in C3, so you won’t lose anything).

  1. Stop your container: Jupyter.make().stop()
  2. Start a new container by going to the url https://<name-of-my-environment>/jupyter/

Comments:

  1. You can increase the memory at most to the memory of the instance you use (typically 32GB) if you need more, ask your system admin to change the instance type.
  2. If you want to change the default size of containers for all the people in your tag: add a 3rd argument to the setConfigValue function:
AwsEcsTaskDefinition.make({
    "id": "ecstd-jupyter"
}).setConfigValue('containerDefinitions', containerConfig, ConfigOverride.TAG);