Jim Cheung

Friday, March 01, 2019

setup a jupyter notebook on local k8s

same as running in docker, I can access it anytime, and keep don't need to worry about too much pip install polluting the system:

Dockerfile:

from ubuntu:bionic

RUN apt-get update && apt-get install -y python3 python3-pip

RUN pip3 install jupyter pandas requests matplotlib

WORKDIR /data

CMD ["/usr/local/bin/jupyter", "notebook", "--ip=0.0.0.0", "--port=9000", "--allow-root"]

to persist data, docker is easy, just mount the local directory as volume, but on k8s it requires two objects: PersistentVolume and PersistentVolumeClaim, one to declare storage and one for requesting storage

kind: PersistentVolume
apiVersion: v1
metadata:
  name: jupyter-data
  labels:
    type: local
spec:
  storageClassName: manual
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/local/data/jupyter-data"

--

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: jupyter-data-pv-claim
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

then in the pod config, add under .spec.volumes:

spec:
  volumes:
    - name: jupyter-data-pv-storage
      persistentVolumeClaim:
        claimName: jupyter-data-pv-claim

and under .spec.containers:

spec:
  containers:
    volumeMounts:
      - mountPath: "/data"
        name: jupyter-pv-storage

start the pod and service and done.

Wednesday, March 13, 2019

list images from a private docker registry:

$ curl registry-host:5000/v2/_catalog
$ curl registry-host:5000/v2/[image_name]/tags/list

quickly create a redis service:

$ kubectl create deployment redisdb --image=redis:latest
$ kubectl expose deployment redisdb --port=26379 --target-port=6379 --external-ip=192.168.33.20 --type=NodePort

don't feel like reading at all, now just reading Stream Processing with Apache Spark

but I'm waiting for Clojure: The Essential Reference

there's always new thing to learn about vim:

e How I revamped my Vim setup

vim

when using fzf Files, it based on current path, add this function to make it based on git project root:

function! s:find_git_root()
  return system('git rev-parse --show-toplevel 2> /dev/null')[:-2]
endfunction
command! ProjectFiles execute 'Files' s:find_git_root()

Tuesday, March 19, 2019

some links about python:

the last one uses Python with Context Managers, which is interesting

joined oreilly's istio online training, here's course materials:

Wednesday, March 27, 2019

convert .m3u8 stream to mp3:

$ ffmpeg -i url/to/file.m3u8 -c copy out.mkv
$ ffmpeg -i out.mkv -c:a libmp3lame out.mp3
Blog Archive