Lesson 29: Dashboards

---

import pandas as pd
import numpy as np
import scipy.stats
import skimage.io

import bootcamp_utils

import colorcet

import bokeh.plotting
import bokeh.io

import holoviews as hv

import panel as pn
pn.extension()

import bokeh_catplot

hv.extension('bokeh')

bokeh.io.output_notebook()

Loading BokehJS ...

---

Note: This notebook contains interactive plots. Full interactivity is not present in the HTML rendering of this notebook. This is because a Python engine needs to be running to update the plots. You can make dashboards that will run in other user’s browsers if you serve it and have the Python engine running on the server side. We will not cover this more advanced feature in the bootcamp.

We have seen that Bokeh allows interactivity in plots. You can zoom and hover over data points to get more information. Bokeh has capabilities beyond that we have not explored. We also saw that we can use HoloViews to create lay out plots and create dropdown menus and sliders to manipulate which data are displayed on plots. (Note that I did not set the HoloViews defaults in using bootcamp_utils because in this lesson we want more fine-grained control over plotting options.)

Dashboarding involves constructing layouts of plots with interactivity, even beyond what we have seen so far. We can do more than just select which data we want to view; we can also trigger any calculation we wish based on mouse clicks or entered text within a graphic.

Panel has emerged as an excellent tool for dashboarding, and we will use it here. We will start with a simple exploration of how parameters affect a function.

A simple example

Let’s start by plotting the PDF of the normal distribution. When we plot it, though, we will write a function to generate the plot and then call the function. This is useful, since we will want to use the function again. (For convenience, I also define a dictionary of options to pass to HoloViews for plotting the curve.)

opts = dict(show_grid=True, frame_height=200, frame_width=350, color="#1f77b3")


def plot_normal_pdf(mu=0, sigma=1):
    x = np.linspace(-10, 10, 200)
    y = scipy.stats.norm.pdf(x, loc=mu, scale=sigma)

    return hv.Curve(data=(x, y), kdims=["x"], vdims=["f(x ; μ, σ)"]).opts(
        **opts
    )


plot_normal_pdf(0, 1)

Data type cannot be displayed:

Looks good, but what if we want to examine how the PDF changes with μ and σ? We could keep plotting it over and over, manually changing the values of µ and σ. Much more instructive would be to create sliders where we can change the values of the parameters and instantaneously see how the plot changes. We can use Panel to create interactive sliders using the FloatSlider widget. To code below implements a simple dashboard, and I comment on the syntax immediately after.

mu_slider = pn.widgets.FloatSlider(
    name="µ", start=-5, end=5, step=0.1, value=0
)
sigma_slider = pn.widgets.FloatSlider(
    name="σ", start=0.1, end=5, step=0.1, value=1
)


@pn.depends(mu_slider.param.value, sigma_slider.param.value)
def plot_normal_pdf(mu=0, sigma=1):
    x = np.linspace(-10, 10, 200)
    y = scipy.stats.norm.pdf(x, loc=mu, scale=sigma)

    return hv.Curve(data=(x, y), kdims=["x"], vdims=["f(x ; μ, σ)"]).opts(
        **opts
    )


widgets = pn.Column(
    pn.Spacer(height=30),
    mu_slider,
    pn.Spacer(height=15),
    sigma_slider,
    width=200,
)
pn.Row(plot_normal_pdf, pn.Spacer(width=15), widgets)

Data type cannot be displayed:

Data type cannot be displayed:

Let’s go through each component. First, we define our widgets, mu_slider and sigma_slider. When building more complicated dashboards, we can look at the Panel documentation to choose which widgets we want to use.

Next, we define our plotting function, plot_normal_pdf(). Here we use Holoviews, but we could use Bokeh (or even Matplotlib or Altair). Notice the @pn.depends function decorator. This links the input from the widget to the computation in the function, so every time we change the interactive widget, the output of the function updates. (We will not discuss decorators in the bootcamp. For this dashboarding application is suffices to know that using the @pn.depends decorator links up the parameter values in the input of the function to the values of the sliders.)

Finally, we set the layout of our dashboard. We can define rows and columns through pn.Row and pn.Column respectively. We can set their heights and widths and add spaces through pn.Spacer. You may have to play around a bit to get it in the format that looks best to you.

Using Panel to explore parameters

Recall from Exercise 7.4 that we investigated the fold change in gene expression as a function of repressor copy number \(R\) and inducer concentration \(c\). The theoretical function, based on an MWC model, was

\begin{align} \text{fold change} = \left[1 + \frac{\frac{R}{K}\left(1 + c/K_\mathrm{d}^\mathrm{A}\right)^2}{\left(1 + c/K_\mathrm{d}^\mathrm{A}\right)^2 + K_\mathrm{switch}\left(1 + c/K_\mathrm{d}^\mathrm{I}\right)^2}\right]^{-1}. \end{align}

There are quite a few parameters here.

Parameter	Description
\(K_\mathrm{d}^\mathrm{A}\)	dissoc. const. for active repressor binding IPTG
\(K_\mathrm{d}^\mathrm{I}\)	dissoc. const. for inactive repressor binding IPTG
\(K_\mathrm{switch}\)	equil. const. for switching active/inactive
\(K\)	dissoc. const. for active repressor binding operator
\(R\)	number of repressors in cell

This is a complicated function of these parameters, and we might want to see how the fold change vs. inducer concentration curve varies based on various parameter values. Dashboarding comes in very handy for this kind of application.

To build our dashboard, we start by defining functions to compute the fold change as a function of the IPTG concentration and the parameters.

def bohr_parameter(c, R, K, KdA, KdI, Kswitch):
    """Compute Bohr parameter based on MWC model."""
    # Big nasty argument of logarithm
    log_arg = (1 + c / KdA) ** 2 / (
        (1 + c / KdA) ** 2 + Kswitch * (1 + c / KdI) ** 2
    )

    return -np.log(R / K) - np.log(log_arg)


def fold_change(c, R, K, KdA, KdI, Kswitch):
    """Compute theoretical fold change for MWC model."""
    return 1 / (1 + np.exp(-bohr_parameter(c, R, K, KdA, KdI, Kswitch)))

Next, we define our sliders. As we explore this function, we would like the parameter to vary on a logarithmic scale. Panel currently does not allow for logarithmic scale on sliders, so we have to specify the parameters as being the logarithm of the parameters.

log_R_slider = pn.widgets.FloatSlider(
    name="log₁₀ R (1/cell)", start=0, end=3, step=0.1, value=2
)
log_K_slider = pn.widgets.FloatSlider(
    name="log₁₀ K (1/cell)", start=-6, end=3, step=0.1, value=0
)
log_KdA_slider = pn.widgets.FloatSlider(
    name="log₁₀ KdA (1/mM)", start=-6, end=3, step=0.1, value=-2
)
log_KdI_slider = pn.widgets.FloatSlider(
    name="log₁₀ KdI (1/mM)", start=-6, end=3, step=0.1, value=-2
)
log_Kswitch_slider = pn.widgets.FloatSlider(
    name="log₁₀ Kswitch", start=-3, end=6, step=0.1, value=1,
)

We can now write a function to generate a plot, given the parameters. We have to use the @pn.depends() decorator to

@pn.depends(
    log_R_slider.param.value,
    log_K_slider.param.value,
    log_KdA_slider.param.value,
    log_KdI_slider.param.value,
    log_Kswitch_slider.param.value,
)
def plot_curve(log_R, log_K, log_KdA, log_KdI, log_Kswitch):
    params = 10.0 ** np.array([log_R, log_K, log_KdA, log_KdI, log_Kswitch])
    c = np.logspace(-6, 2, 200)

    opts = dict(
        frame_height=250,
        frame_width=350,
        logx=True,
        show_grid=True,
        xlabel="[IPTG] (mM)",
        ylabel="fold change",
        ylim=(-0.05, 1.05),
        color="#1f77b3",
    )

    return hv.Curve((c, fold_change(c, *params))).opts(**opts)

Finally, we can lay out our dashboard and explore the function.

pn.Row(
    plot_curve,
    pn.Spacer(width=15),
    pn.Column(
        log_R_slider,
        log_K_slider,
        log_KdA_slider,
        log_KdI_slider,
        log_Kswitch_slider,
        width=200,
    ),
)

Data type cannot be displayed:

Data type cannot be displayed:

In playing with the slider, we see that a difference between \(K_\mathrm{d}^\mathrm{A}\) and \(K_\mathrm{d}^\mathrm{I}\) is required to get repression. As we would expect, we need \(K_\mathrm{d}^\mathrm{I} < K_\mathrm{d}^\mathrm{A}\) in order to get more repression with increasing IPTG concentration.

The effects of the other parameters are more complicated and interdependent, but can nonetheless be explored by varying the sliders.

Bacterial growth

In auxiliary lessons, we will explore image processing. Here, we will make a dashboard to display images in a time lapse. We will display a time lapse movie of growing Bacillus subtilis cells, acquired by Jin Park from the Elowitz lab. The image are stored in files named like data/bacterial_growth/bacillus_001.tif, for a total of 55 frames. To load an image, we use skimage.io.imread().

im = skimage.io.imread('data/bacterial_growth/bacillus_001.tif')

This stores the image as a Numpy array. To display the image, we can use HoloView’s Image Element. Before using that, we need to set up some of the image’s dimensions first. To get the scale of the axes right, we need to know the interpixel distance. From the metadata provided by Jin Park, the interpixel distance is 64.5 nanometers.

ip_distance = 0.0645

Since we’re doing a time lapse, we should also know the time between frames. In this case, it was 15 minutes.

dt = 15

To get the aspect ratio correct, we need to specify the frame width we want, and then set the height accordingly.

frame_width = 200
frame_height = int(frame_width * im.shape[0] / im.shape[1])

We can now set the bounds kwarg for our call to hv.Image().

bounds = [0, 0, im.shape[1]*ip_distance, im.shape[0]*ip_distance]

Now, we’re ready to plot the image.

hv.Image(im, bounds=bounds).opts(
    xlabel="µm",
    ylabel="µm",
    title="t = 0 min",
    frame_width=frame_width,
    frame_height=frame_height,
    cmap='viridis',
)

Data type cannot be displayed:

Be default, we are displaying the image with a Viridis colormap, which goes from purple for low intensity to yellow for high. This default was set when we called bootcamp_utils.hv_defaults.set_defaults(), and is a good perceptual default colormap.

Now let’s build our dashboard. We want a slider to switch from frame to frame and also a pulldown menu that allows us to switch colormaps. Because frame numbers are integers, we use an IntSlider instead of a FloatSlider. For the color map, we use a Select widget.

frame_slider = pn.widgets.IntSlider(name="frame", start=1, end=55, value=1)
colormap_selector = pn.widgets.Select(
    name="colormap",
    options=["gray", "fire", "magma", "viridis"],
    value="viridis",
)


@pn.depends(frame_slider.param.value, colormap_selector.param.value)
def show_bacillus(frame, cmap):
    # Load in appropriate image
    fname = "data/bacterial_growth/bacillus_{frame:03d}.tif".format(
        frame=frame
    )
    im = skimage.io.imread(fname)

    return hv.Image(im, bounds=bounds).opts(
        xlabel="µm",
        ylabel="µm",
        title=f"t = {dt*(frame-1)} min",
        frame_width=frame_width,
        frame_height=frame_height,
        cmap=cmap,
    )


pn.Row(
    show_bacillus,
    pn.Spacer(width=15),
    pn.Column(
        pn.Spacer(height=30),
        frame_slider,
        pn.Spacer(height=15),
        colormap_selector,
    ),
)

Data type cannot be displayed:

Data type cannot be displayed: