Jim Cheung

Friday, March 01, 2019

setup a jupyter notebook on local k8s

same as running in docker, I can access it anytime, and keep don't need to worry about too much pip install polluting the system:


from ubuntu:bionic

RUN apt-get update && apt-get install -y python3 python3-pip

RUN pip3 install jupyter pandas requests matplotlib


CMD ["/usr/local/bin/jupyter", "notebook", "--ip=", "--port=9000", "--allow-root"]

to persist data, docker is easy, just mount the local directory as volume, but on k8s it requires two objects: PersistentVolume and PersistentVolumeClaim, one to declare storage and one for requesting storage

kind: PersistentVolume
apiVersion: v1
  name: jupyter-data
    type: local
  storageClassName: manual
    storage: 1Gi
  • ReadWriteOnce hostPath: path: "/local/data/jupyter-data" -- kind: PersistentVolumeClaim apiVersion: v1 metadata: name: jupyter-data-pv-claim spec: storageClassName: manual accessModes: - ReadWriteOnce resources: requests: storage: 1Gi

then in the pod config, add under .spec.volumes:

  • name: jupyter-data-pv-storage persistentVolumeClaim: claimName: jupyter-data-pv-claim

and under .spec.containers:

  • mountPath: "/data" name: jupyter-pv-storage

start the pod and service and done.

Wednesday, March 13, 2019

list images from a private docker registry:

$ curl registry-host:5000/v2/_catalog
$ curl registry-host:5000/v2/[image_name]/tags/list

quickly create a redis service:

$ kubectl create deployment redisdb --image=redis:latest
$ kubectl expose deployment redisdb --port=26379 --target-port=6379 --external-ip= --type=NodePort

don't feel like reading at all, now just reading Stream Processing with Apache Spark

but I'm waiting for Clojure: The Essential Reference

there's always new thing to learn about vim:

e How I revamped my Vim setup


when using fzf Files, it based on current path, add this function to make it based on git project root:

function! s:find_git_root()
  return system('git rev-parse --show-toplevel 2> /dev/null')[:-2]
command! ProjectFiles execute 'Files' s:find_git_root()

Tuesday, March 19, 2019

some links about python:

the last one uses Python with Context Managers, which is interesting

joined oreilly's istio online training, here's course materials:

Wednesday, March 27, 2019

convert .m3u8 stream to mp3:

$ ffmpeg -i url/to/file.m3u8 -c copy out.mkv
$ ffmpeg -i out.mkv -c:a libmp3lame out.mp3
Blog Archive