Skip to main content

Notebook issues

Table of Contents​

Installing pyodbc within a notebook​

Pyodbc is a module that allows you to connect to multiple ODBC databases such as Google cloud Bigquery, Azure SQL,...

In order to install it, you should use a custom Dockerfile as pyodbc requires additional packages to be installed, which are not by default present in our notebook image. If you need more information on how to use a custom Dockerfile, take a look at the notebook reference.

Our base image starts from the latest Ubuntu LTS version. Microsoft describes the steps that are required to install the odbc driver here The following lines should be added to your Dockerfile before installing pyodbc:

USER root
RUN apt-get update && \
    apt-get install -y curl gnupg2 && \
    curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - && \
    curl https://packages.microsoft.com/config/ubuntu/20.04/prod.list > /etc/apt/sources.list.d/mssql-release.list && \
    apt-get update && \
    ACCEPT_EULA=Y apt-get install -y unixodbc-dev msodbcsql17 && \
    apt-get autoremove -y curl gnupg2 && \
    apt-get clean && rm -rf /var/lib/apt/lists/*

USER jovyan

Adding a Jupyter extension to the notebooks​

To add Jupyter extensions to notebooks, we recommend to use our custom dockerfile support.

To get started, first create the template for the custom Dockerfile.

mkdir -p notebooks
conveyor notebook export -f notebooks/Dockerfile

This will produce a Dockerfile that looks something like this:

FROM {{ .BaseNotebookImage }}
...

#If you want to install custom notebook extensions you can do so like the following
# RUN pip install jupyterlab-spellchecker

...

Most of the context is skipped in the example, but the most important part is kept. An example on how to add a custom notebook extension. In this case we shall install the jupyterlab-spellchecker extension, this extension will spell check the markdown cells of your notebooks.

To install it, we only have to enable the following line:

RUN pip install jupyterlab-spellchecker

By removing the # the line will be activated when building the container.

Using the custom Dockerfile​

Next, we have to make sure the notebook dockerfile is picked up. To do this first create a notebooks.yaml.

conveyor notebook create

When the question is asked:

? Provide custom Dockerfile path

Answer with notebooks/Dockerfile.

? Provide custom Dockerfile path notebooks/Dockerfile

This will result in a notebooks.yaml that looks like:

notebooks:
- name: default
  customDockerfile: notebooks/Dockerfile

If you already had a notebooks.yaml you can just add the customDockerfile yourself.

Testing it out​

The notebook that is just launched will have your extension installed! To test it out start a new notebook. Change the cell type of the first cell to markdown and write:

I mistoke this word for something else

You will see the word mistoke highlighted in red. This means the extension is working.

Using private dependencies in notebooks​

You would like to provision project notebooks with packages from your companies private source control repository. In this guide we assume you have added the private repo to the project requirements.txt and are able to install the dependency using your personal ssh key.

argparse==1.4.0
requests==2.26.0
...
git+ssh://git@github.com/MY_ORGANIZATION/private_repo.git
...

1. Create a custom notebook Dockerfile​

conveyor notebook export --file DockerfileNotebook

2. Update the Dockerfile​

Update the file created in the previous step. Use your custom source control domain when using anything else then github.com.

...

# Install ssh client and git
USER root
RUN apt-get update -y && apt-get install -y ssh-client git

# Download public key for your ssh connection (using github.com as an example)
USER jovyan
RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts

...

# install dependencies and src code in editor mode
WORKDIR /home/jovyan/work/conveyor_project
COPY --chown=jovyan:users requirements.txt requirements.txt
COPY --chown=jovyan:users setup.py setup.py
RUN --mount=type=ssh,mode=777 python -m pip install --no-cache-dir -r requirements.txt
...

3. Create a new notebook​

Create a new notebook using the CLI and pass the name of the adapted Dockerfile when requested.

conveyor notebook create --name mynotebook --ssh default