Jim Cheung

Thursday, May 03, 2018

k8s, k8s everywhere:

and google is slowly killing docker, by open sourcing new build image tool and container runtime:

a good article on unprivileged container: Towards unprivileged container builds

I think I'll just watch, tools are changing so fast in this area ...

right now I'm more interested in graalvm:

so, pandas for everyone?

I didn't pay much attention to it, until I watched this talk: Scaling Uber's Elasticsearch Clusters

they allows user input sql to query elasticsearch, a query proxy parses the query and transforms to elasticsearch syntax, (also routing, rate limiting, caching and workload management)

the query parser mentioned should be queryparser

Using SQL to query Kafka, MongoDB, MySQL, PostgreSQL and Redis with Presto

this is so cool, query all different data sources in one place

more on presto: Engineering Data Analytics with Presto and Parquet at Uber

finally some links

Sunday, May 06, 2018

reading few books at the same time (from local library):

tried few micro emacs editors:

also mcedit can switch to emacs keybindings:

$ cp /etc/mc/mc.emacs.keymap ~/.config/mc/mc.keymap

they work almost the same, I'll stick with mcedit

good introduction on gvisor: Sandboxing Docker with Google's gVisor

Importantly, gVisor does not simply redirect application system calls through to the host kernel. Instead, gVisor implements most kernel primitives (signals, file systems, futexes, pipes, mm, etc.) and has complete system call handlers built on top of these primitives.

and it shares same network stack with fuchsia: go/src/netstack

so it's an os written in go for desktop, container and protentially app?

here's a nice overview on chroot, cgroups and namespaces

clojure has more experiment works on the tools.deps: add-lib

Rich Hickey interviewed in a podcast

looks like he's working on a new version of Datomic/codeq: Creates Datomic dbs from git repos

I am always interested in writing a proxy:

Hosting multiple apps on the same server — Implement a reverse proxy with Node

I still don't have time to upgrade my lxd to 3.0 yet

here's a guide for the cluster mode: LXD Clusters: A Primer

Using Kubernetes for Local Development

it uses GoogleContainerTools/skaffold: Easy and Repeatable Kubernetes Development

Monday, May 07, 2018

use Paramiko to execute remote commands:

import paramiko
import os

client = paramiko.SSHClient()
client._policy = paramiko.WarningPolicy()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())

client.load_system_host_keys()

ssh_config = paramiko.SSHConfig()
with open(os.path.expanduser('~/.ssh/config')) as f:
    ssh_config.parse(f)

host_data = ssh_config.lookup('stg-data')

cfg = {'hostname': host_data['hostname'],
       'username': host_data['user'],
       'pkey': paramiko.ed25519key.Ed25519Key(filename=host_data['identityfile'][0])}

client.connect(**cfg)

stdin, stdout, stderr = client.exec_command('ls /tmp/')
for line in stdout:
    print(line)

client.close()

above reads '~/.ssh/config for host ip and supports Ed25519` key

this hackernews thread brings me back to eshell:

~/.eshell/profile for startup scripts and ~/.eshell/alias for alias

for loop examples:

for f in web1 web2 { scp info.php $f:/var/www/html/oscmax/catalog }
for f in {ls -1d *} {echo $f; svn up $f}
for f in {find . -name '*.java'} {mv $f (replace-regexp-in-string "\\.java$" ".scala" f)}
# Those parens may also be curly braces. Paren for lisp, curly for subshell.
# See how $f is used in curly brace but f is used in parens
for f in db1 db2 {mysqldump -d -uuser -ppwd (concat "prefix_" f "_suffix") >(concat "/tmp/backup_" f ".sql")}

this eshell prompt is very cool: Making eshell your own

Thursday, May 17, 2018

finished reading Think Data Structures: Algorithms and Information Retrieval in Java, it's good and I'm planning to implement examples in clojure

still reading Effective Java (3rd Edition), it's such a great book. I'm have to buy a copy.

borrowed another book from local library:

another book but not computer related: History of Pictures: From the Cave to the Computer Screen, about drawing, painting, lens and photography, great book.

Saturday, May 26, 2018

it's GDPR day, mozilla has a good article on it: 13 things to know about the GDPR

I think it's a good thing, it could help us to understand what a free service really is

while reading this interesting tool: dgsh - directed graph shell

I discovered couple more:

Apache Flink 1.5.0 Release Announcement

I've been reading some flink documentations, now it's time to upgrade: Apache Flink 1.5-SNAPSHOT Documentation

here's an article on different stream processing system: Spark Streaming vs Flink vs Storm vs Kafka Streams vs Samza : Choose Your Stream Processing Framework

I saw only good things on flink

I read last chapter on Kafka: The Definitive Guide: Real-Time Data and Stream Processing at Scale, about its stream api

but I need to see more examples ...

some about reasonml

Sunday, May 27, 2018

I hate scrum, but the question is if not scrum then what?

Reality Driven Development: Fixing Project Management in Software

Kanban is an ideal software project methodology when you remove accountability concerns from the equation. The best developers I know are obsessed with solving problems. If they are pulled in too many directions, when the end of the day comes they'll just turn their brain off and mentally check out for the evening.

If they are focused on a single problem...they will obsess about that single problem after hours. They will sleep on it. They'll be distracted while watching TV. They'll run to the computer at 10pm to test an idea that they had about the problem.

As a business owner, you want those extra mental hours spent solving your problem. At the same time, you have to identify when you have people who do this for you and insistently make them take extra time off every now and then. Whether for their family, their kids or their health the last thing you want to do is burn out an asset like that by letting a person push themselves hard enough to create problems outside of work.

a little bit scary for me (as a developer)

at the end author suggested kanban and light pairing, which is make sense to me

this is also good, Great PMs don’t spend their time on solutions

spend more time to define the problem.

another good one: You’re Doing Scrum Wrong, and Here’s How to Fix It

it introduces spike (from extreme programming I believe), which is much much better than sprint zero

Spot A Fake Agile Team In Under 7 Minutes

you asks two questions:

  1. How is feedback gathered?
  2. How much budgets is available for changes?

make sense, it's all about the iteration

I actually learned few things by reading all these articles recently:

first one, agile manifesto said:

Individuals and interactions over processes and tools

so if scrum doesn't work out for your team, drop it, don't force to fit into the framework

the other thing is deadline

you can try search the scrum guide, you won't able to find deadline in it.

no deadline is too good to be true, so how do scrum deal with it?

there's only one way - cut features, which means even requirements can be changed and should adapt agile

however it's same as no deadline, too good to be true.

so any scrum guide that doesn't cover how to deal with deadline is bullshit.

finally, the best comes from this comment: Agile often allows no time for doing it right, leading to high levels of technic...

Agile is also totally unsuited for innovation. You will never get a Xerox Parc or SRI with agile. You can only make what you already know to make with agile.

that sums up my thoughts on scrum

I'm following these guides for some experiment:

ideas are from uber's blogs:

also evaluating linkedin/pinot: A realtime distributed OLAP datastore

and compare it with clickhouse and druid: Comparison of the Open Source OLAP Systems for Big Data: ClickHouse, Druid and Pinot

learned something new with emacs:

m-x ffap: open file at point, (short name for find-file-at-point)

ibuffer

filters:

once filter group created, use /g to name it, // to clear filters

sort:

misc:

bookmark

mark

occur

in the *Occur* buffer:

repeat

rgrep

dired

register

Tuesday, May 29, 2018

Emacs 26.1 released

I compiled with Mitsuharu Yamamoto's fork

after compiled, it shows two errors:

also change command key as meta:

(setq mac-command-modifier 'meta)
Blog Archive