jchk - /blog/2018-12

Sunday, December 3, 2018

continue to watch clojure conj 2018 videos:

even though the tool REBL itself is not open sourced, I think datafy and nav (added to clojure v1.10.0 RC 2) could bring the whole repl experience close to jupyter notebook, and even better (you can nav data)

and a very fun talk: Every Clojure Talk Ever - Alex Engelberg and Derek Slager

Thursday, December 6, 2018

found a much easier way to access google drive files from google colaboratory:

from google.colab import drive
drive.mount('/content/drive')

however, it's better to chdir to the folder before you feed any files to pandas etc, since the space in My Drive is quite tricky:

import os
import pandas as pd
os.chdir("/content/drive/My Drive")
!ls
pd.read_csv('file.csv')

saw few news book from no starch press on safari books online:

but I'm reading Bad Blood: Secrets and Lies in a Silicon Valley Startup, it's also in 5 books I loved in 2018 of Bill Gates

Tuesday, December 18, 2018

I'm not a fan of mixing server and client codes together (server-side rendering, SSR), pretty complicated without good reason.

anyway, there's a redis package needs to exclude from client side build, according to this article: SSR and Server Only Modules

found the simplest way to do it, just add the browser field to package.json:

"browser": {
  "redis": false
}

Clojure 1.10 release, heard the error reporting is much better, more details here: Clojure 1.10 error messages – Inside Clojure

under emacs, I have a function to crawl url and return markdown style link: [title](url], I want something easy under vim too

first need to extract title tag from target url, I found this tool is easy to install and use: ericchiang/pup: Parsing HTML at the command line

then a simple script to ouptut markdown code:

#!/bin/bash

if [ -z $1 ]; then
    echo "usage: $0 [url]"
    exit 1
fi

URL="$1"
TITLE=$(curl -s $URL | pup 'title text{}')

echo "[$TITLE]($URL)"

in vim just call !!mdlink url to insert markdown code

Wednesday, December 19, 2018

an interesting talk about lisp history: A tale of LISP (slides), by Renzo Borgatti author of Clojure The Essential Reference

some references from the talk:

another interesting talk on lisp: Lessons Learned Implementing Common Lisp with LLVM

listened two episodes of Matt Klein talks about envoy:

and also EnvoyCon 2018 videos

Thursday, December 20, 2018

found few useful python tools from Python Bytes Podcast:

jtpio/jupyterlab-python-bytecode: JupyterLab extension to inspect Python Bytecode, display bytecode in notebook, very cool, requires jupyterlab
pyjanitor: Python implementation of the R package janitor, and more, cleaning pandas dataframes
santinic/pampy: Pampy: The Pattern Matching for Python you always dreamed of.
sixty-north/asq: asq is simple implementation of a LINQ-inspired API for Python which operates over Python iterables

slowly reading Practical Binary Analysis

also found a new release book from safari books online: Microservices Patterns

watching Algorithms: 24-part Lecture Series by Robert Sedgewick and Kevin Wayne as well

Monday, December, 24, 2018

tmux set different background color for active and inactive panes:

set -g window-style 'fg=colour247,bg=colour236'
set -g window-active-style 'fg=colour250,bg=black'

and to apply config without restart tmux (hit prefix key first) :

:source-file ~/.tmux.conf

keys for vimdiff:

do: diff obtain
dp: diff put
[c: previous difference
]c: next difference
:diffupdate: diff update
:syntax off: syntax off
zo: open folded text
zc: close folded text

Merry Christmas!

Tuesday, December 25, 2018

found a new font for terminal: Merchant Copy Font

getting start with envoy

create envoy.yaml:

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address: { address: 0.0.0.0, port_value: 9901 }
static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address: { address: 0.0.0.0, port_value: 10000 }
    filter_chains:
    - filters:
      - name: envoy.http_connection_manager
        config:
          stat_prefix: ingress_http
          codec_type: AUTO
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match: { prefix: "/" }
                route: { host_rewrite: www.google.com, cluster: service_google }
          http_filters:
          - name: envoy.router
  clusters:
  - name: service_google
    connect_timeout: 0.25s
    type: LOGICAL_DNS
    # Comment out the following line to test on v6 networks
    dns_lookup_family: V4_ONLY
    lb_policy: ROUND_ROBIN
    hosts: [{ socket_address: { address: google.com, port_value: 443 }}]
    tls_context: { sni: www.google.com }

create Dockerfile:

FROM envoyproxy/envoy:latest
COPY envoy.yaml /etc/envoy/envoy.yaml

build:

$ docker build -t envoy:v1 .

run:

$ docker run -p 9901:9901 -p 10000:10000 envoy:v1

test:

$ curl -v localhost:10000

now deep dive into Envoy v2 APIs

Thursday, December 27, 2018

was going to install minikube, but tried following this instruction to install kubernetes on ubuntu 18.04, it works~

prepare vagrant, add following lines:

vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
vb.customize ["modifyvm", :id, "--natdnsproxy1", "on"]

steps as below:

install docker:
```
$ sudo apt install docker.io
```
add kubernetes apt source

install kubeadm, kubelet:

$ sudo apt install -y kubeadm kubelet kubernetes-cni

make sure swap is off:

$ sudo swapoff -a
$ sudo swapon --summary

use kubeadm to setup the master (change ip address here):

$ sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.33.10

setup kubectl:

$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

setup network (for vagrant you need to pass --iface=enp0s8 argument to flanneld):

$ curl -OL https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

# update kube-flannel.yml, add the --iface line to args:

      containers:
      - name: kube-flannel
        image: quay.io/coreos/flannel:v0.11.0-amd64
        command:
        - /opt/bin/flanneld
        args:
        - --ip-masq
        - --kube-subnet-mgr
        - --iface=enp0s8

# then apply it
$ kubectl apply -f kube-flannel.yml

untaint master:

$ kubectl taint nodes --all node-role.kubernetes.io/master-

test:

$ kubectl get all --namespace=kube-system

dashboard

just follow steps on Kubernetes Dashboard repo

$ kubectl proxy

and tunnel to host

$ ssh -L 8001:localhost8001 kube-master

Satursday, December 29, 2018

end of 2018, here's summary of this year's readings:

|---------+-------------------------------------------------------------------------------------|
| 2018-01 | Streaming Systems: The What, Where, When, and How of Large-Scale Data Processing    |
|---------+-------------------------------------------------------------------------------------|
| 2018-03 | Professional Clojure                                                                |
|---------+-------------------------------------------------------------------------------------|
| 2018-04 | Ed Mastery                                                                          |
|---------+-------------------------------------------------------------------------------------|
| 2018-05 | Think Data Structures: Algorithms and Information Retrieval in Java                 |
|         | Effective Java (3rd Edition)                                                        |
|         | Designing for Scalability with Erlang/OTP: Implement Robust, Fault-Tolerant Systems |
|         | Think Data Structures: Algorithms and Information Retrieval in Java                 |
|         | A Common-Sense Guide to Data Structures and Algorithms                              |
|---------+-------------------------------------------------------------------------------------|
| 2018-06 | Stream Processing with Apache Flink                                                 |
|         | Learning Spark                                                                      |
|         | Kafka: The Definitive Guide                                                         |
|         | Kubernetes in Action                                                                |
|         | Microservices with Clojure                                                          |
|         | Getting Clojure                                                                     |
|---------+-------------------------------------------------------------------------------------|
| 2018-07 | Rework                                                                              |
|         | Pandas for Everyone                                                                 |
|         | Different: Escaping the Competitive Herd                                            |
|         | React Quickly: Painless web apps with React, JSX, Redux, and GraphQL                |
|         | Vue.js 2 Design Patterns and Best Practices                                         |
|         | Vue.js: Up and Running                                                              |
|---------+-------------------------------------------------------------------------------------|
| 2018-08 | Programming Clojure 3rd Edition                                                     |
|---------+-------------------------------------------------------------------------------------|
| 2018-09 | A Philosophy of Software Design                                                     |
|         | Practical ES6                                                                       |
|         | Optimizing Java                                                                     |
|         | Pro Vue.js 2                                                                        |
|         | Hacking: The Art of Exploitation, 2nd Edition                                       |
|         | Google Cloud Platform in Action                                                     |
|         | Get Programming with Go                                                             |
|         | Managing Kubernetes                                                                 |
|---------+-------------------------------------------------------------------------------------|
| 2018-11 | OAuth 2 in Action                                                                   |
|         | Designing Web APIs                                                                  |
|         | Refactoring, 2nd Edition                                                            |
|---------+-------------------------------------------------------------------------------------|
| 2018-12 | Practical Binary Analysis                                                           |
|         | Serious Python                                                                      |
|         | Math Adventures with Python                                                         |
|         | Bad Blood: Secrets and Lies in a Silicon Valley Startup                             |
|         | Microservices Patterns                                                              |
|---------+-------------------------------------------------------------------------------------|

Blog Archive