Version v0.3 of the documentation is no longer actively maintained. The site that you are currently viewing is an archived snapshot. For up-to-date documentation, see the latest version.
Allocate memory, CPU, GPU, or other resources according to your need (1 CPU and 2Gi of Memory are good starting points)
kubectl get nodes "-o=custom-columns=NAME:.metadata.name,GPU:.status.allocatable.nvidia\.com/gpu"
Extra Resource Limits
section: {"nvidia.com/gpu": "1"}
Click Spawn
The images are 10’s of GBs in size and can take a long time to download depending on your network connection
You can check the status of your pod by doing
kubectl -n ${NAMESPACE} describe pods jupyter-${USERNAME}
GKE users if you have IAP turned on the pod will be named differently
jupyter-accounts-2egoogle-2ecom-3USER-40DOMAIN-2eEXT
You should now be greeted with a Jupyter Notebook interface.
The image supplied above can be used for training Tensorflow models with Jupyter. The images include all the requisite plugins, including Tensorboard that you can use for rich visualizations and insights into your models.
To test the install, we can run a basic hello world (adapted from mnist_softmax.py )
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
import tensorflow as tf
x = tf.placeholder(tf.float32, [None, 784])
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))
y = tf.nn.softmax(tf.matmul(x, W) + b)
y_ = tf.placeholder(tf.float32, [None, 10])
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))
train_step = tf.train.GradientDescentOptimizer(0.05).minimize(cross_entropy)
sess = tf.InteractiveSession()
tf.global_variables_initializer().run()
for _ in range(1000):
batch_xs, batch_ys = mnist.train.next_batch(100)
sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})
correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print(sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels}))
Paste the example into a new Python 3 Jupyter notebook and execute the code. This should result in a 0.9014 accuracy result against the test data.
Please note that when running on most cloud providers, the public IP address will be exposed to the internet and is an unsecured endpoint by default. For a production deployment with SSL and authentication, refer to the documentation.
The Jupyter Notebook pods are assigned the jupyter-notebook
service account. This service account is bound to jupyter-notebook
role which has namespace-scoped permissions to the following k8s resources:
This means that you can directly create these k8s resources directly from your jupyter notebook. kubectl is already installed in the notebook, so you can create k8s resources running the following command in a jupyter notebook cell
!kubectl create -f myspec.yaml
If using Jupyter Notebooks on GKE, you can submit docker image builds to Cloud Build which builds your docker images and pushes them to Google Container Registry.
Activate the attached service account using
!gcloud auth activate-service-account --key-file=${GOOGLE_APPLICATION_CREDENTIALS}
If you have a Dockerfile in your current directory, you can submit a build using
!gcloud container builds submit --tag gcr.io/myproject/myimage:tag .
Advanced build documentation for docker images is available here