Jim Cheung

Thursday, February 02, 2017

some good discussion on hackernews regarding The JVM is not that heavy

many people mentioned about slow startup time of clojure, it's biggest pain point for me too.

playing with aerofs/openjdk-trim: Tool to trim unneeded classes from an OpenJDK build., I don't know how to use strace to get what classes have been used ...

but I learn a new strace option:

strace outputs to stderr, so to grep you'll need to:

$ strace program 2>&1 | grep needle

I can get list of loaded java classes by:

$ java -verbose:class program

and I managed to successfully pack a trimmed version of jre:

$ java -verbose:class program | gawk '$2 ~ /^[java|sun]/ {o=gensub(/(.+\.+)(.+)/, "\\1", "g", $2); gsub(/\./, "/", o); print "+ /" o "*"}' | sort | uniq | pbcopy

(yeah awk is useful)

remember to remove jdk lines in linux/files.filter and exclude javafx library files:

- /jre/lib/*/libgstreamer-lite.so
- /jre/lib/*/libjavafx_font_freetype.so
- /jre/lib/*/libjavafx_font_pango.so
- /jre/lib/*/libjavafx_font.so
- /jre/lib/*/libjavafx_iio.so
- /jre/lib/*/libjfxwebkit.so
- /jre/lib/*/libjfxmedia.so

I got a 28M jre, can run my java/clojure programs without problems.

however, startup time for clojure programs still not improved.

they are dynmamic invoked, I guess need more sophisticated tool to trim the clojure library.

Tuesday, February 07, 2017

happy to hear RethinkDB joins The Linux Foundation

and I just started using rethinkdb, saved some notes here

it is good experience, writing queries using chaining methods is very easy.

Thursday, February 09, 2017

borrowed Introducing Go and Infrastructure as Code from local library.

introducing go is a very short book, already finished reading it. I didn't code go for quite a while, so this book is very useful to me, refreshed my memory and it covers the basic stuffs very well.

still slowly reading Learning ClojureScript, actually it is a very good book, highly recommended.

three data tools:

metabase looks simple (just a jar file) and will give it a try.

Sunday, February 12, 2017

trying to write some clojurescript

converted my favourite example:

var jsdom = require("jsdom");
 
jsdom.env({
  url: "http://news.ycombinator.com/",
  scripts: ["http://code.jquery.com/jquery.js"],
  done: function (err, window) {
    var $ = window.$;
    console.log("HN Links");
    $("td.title:not(:last) a").each(function() {
      console.log(" -", $(this).text());
    });
  }
});

to cljs:

#!/usr/local/bin/lumo -q

(def jsdom (js/require "jsdom"))

(-> jsdom
    (.env #js{:url "http://news.ycombinator.com/"
              :scripts #js["http://code.jquery.com/jquery.js"]
              :done (fn [err, window]
                      (let [$ (.-$ window)
                            nodes ($ "td.title:not(:last) a")]
                        (print "HN Links")
                        (-> nodes
                            (.each (fn []
                                     (this-as this
                                       (print "-" (.text ($ this)))))))))}))

I kept jquery here, but it's better to use jayq

to access js object property, use .-name

need to use native js array (use #js) as parameters to :scripts

use this-as this to access this object inside scope

with lumo, I finally can start writing scripts in clojurescript

simple examples like a cron:

(js/setInterval
 (fn [] (print "this run every 5 seconds"))
 5000)

and execute external program:

(def child-process (js/require "child_process"))

(js/child-process.exec "ping -c5 www.google.com"
      (fn [err, stdout, stderr]
        (print stdout)))

maybe I need to re-read Node.js in Practice

borrowed Programming Elixir 1.2 from local library when I saw it today. first elixir book I'm going to read.

When I studied rethinkdb, I choosed ruby to start, I just like its syntax. looks like a right time to start elixir.

Tuesday, February 14, 2017

to read epub file under terminal:

$ brew install ebook-tools

or under ubuntu:

$ sudo apt-get install epub-utils

then

$ einfo -pp book.epub | elinks --dump | less

finished learning clojurescript, it's a good book.

skipped macro chapter as usual, also didn't pay much attention on om stuffs, since I'll use reagent.

need some practice and lots of libraries to try.

Thursday, February 16, 2017

generate 10 MB file filled with random text

$ base64 /dev/urandom | head -c 10000000 > file.txt

network throughput benchmark using iperf3: Network Throughput Benchmarking Linux

Under the Hood: AWS CodeDeploy and Auto Scaling Integration

google announced Cloud Spanner beta

some related reads:

CockroachDB is also based on their spanner paper

I don't really understand this yet, but will find out later:

CockroachDB uses MVCC. Spanner uses 2PL and F1 uses OCC.

also something with MySQL: Vitess is a database clustering system for horizontal scaling of MySQL.

Announcing TensorFlow 1.0

the nice thing is there is a java (and go) sdk:

tensorflow/tensorflow/java

more resources for ML:

terryum/awesome-deep-learning-papers: The most cited deep learning papers

reading Infrastructure as Code

two minimal os mentioned (suitable as docker host):

DigitalOcean announced load balancer: An Introduction to DigitalOcean Load Balancers

not useful for me and the price is high for me ($20), but I found out Floating IPs is useful and costs $0.

just noticed lxd is installed by default on ubuntu 16.04, so I read about the difference between lxd and docker:

What is the difference between Docker, LXD, and LXC

lxd hosts linux instances, docker hosts apps...

which means lxd should be compared with vagrant (virtualbox):

Linux Containers (LXD) as an Alternative to VirtualBox for WordPress Development

this is interesting, using lxd to host linux vm is much faster than vagrant.

still need to read more on networking and file sytem part...

seems lxd also supports kubernetes: The canonical distribution of kubernetes

Saturday, February 18, 2017

learning lxd, will put notes on lxd page.

two related tools to learn as well: Juju and ZFS

some kubernetes shortcomings from this comment: We've been running Kubernetes in production for over a year. It has a steep lear...

  • it is impossible to trigger a rescheduling / rebalancing. When a new node comes in (via AutoScaling policy or whatever), kubernetes doesn't do anything. Thus, the new nodes can be sitting there doing nothing.

  • once a pod has been scheduled onto a node, it never reschedules it anywhere else. The node may be experiencing problems, thus the pod is affected. k8s doesn't do anything to heal that – it could simply delete the pod so it is rescheduled somewhere else.

  • docker itself constantly ships with a lot of bugs. To this day (we are on docker 1.12.6), we constantly have problems with the docker daemon hanging or becoming unresponsive. I'm not sure if > k8s can do much about this, but I feel like it should since we don't directly control docker.

  • doesn't integrate more tightly with the OS / cloud provider. For example, it could perform health checks and decide if the node should be restarted, terminated, or idle.

for me, I'll wait until kubernetes switches to another runner (like runC),

docker tries to do too much and new features always comes with new bugs.

coreos dropped fleet and moving to kubernetes: Container orchestration: Moving from fleet to Kubernetes

this article: Five Months of Kubernetes mentions coreos/kube-aws: CoreOS Kubernetes on AWS, which is a win-win solution for coreos and kubernetes, and a reasonable good choice for existing aws users (it's important that we can access aws services out of the box)

docker should kill swarm and adapt kubernetes

I'm more interested in lxd now and learn specific components, rather than learn how to setup a cluster with magic.

I completely agree with this comment on hackernews: The ecosystem around containerization is still emerging and is in a rapid state ...

tl;dr Don't use either. Learn k8s slowly over the next 2 years as they work out the kinks, since it will play a larger role in the future. In 5 years, you may want to use some of this on an important project, but right now, it's all a joke, and, in general, the companies that are switching now are making a very bad decision.

so true, you need to learn kubernetes, but use it on development.

anyways, I'm learning and dropped some notes on kubernetes

I started with minikube, which start a kubernetes cluster locally using virtualbox

it's simple and easy, you can taste kubernetes immediately with no setups.

the dashboard is just like gce, I still use the same php-redis-guestbook example, which I played on gce a year ago.

I also started a juju locally, setup is not as easy as minikube, but it looks quite powerful as well.

however, playing with it is ok, maintain it (automation) is a different kind of story. I won't even try.

Friday, February 24, 2017

I love lxd, but to expose port to the host is not quite user friendly:

$ iptables -t nat -A PREROUTING -p tcp -i enp0s8 --dport 8080 -j DNAT --to-destination 10.138.36.66:8080

kubernetes's proxy basically is iptables rules too. so I think I'd better learn some iptables ...

played aws ecs, some notes:

from here you can run the task or create a service using the task, also have options to add load balancer and auto scaling to the service

the difference is you can stop a task, but service will maintain a desire number of running tasks

means if you stopped a task which belongs to a service, a new task will be created by the service immediately (like kubernetes)

when an alb is added, you can add service to a target group, and by using listener rules, you can match path pattern and route traffic to the service

personally I like kubernetes more, but aws takes care of many important parts: load balancing, auto scaling, vpc to connect other aws services.

just use gke if you want kubernetes, and there's not reason to use kubernetes if you're on aws

still reading Programming Elixir 1.2, I found quite a few concepts are similar to clojure, which is nice and easy for me.

like the pipe operator |>, it's same as clojure's thread first macro: ->

I like it separates Enum (eager) and Stream (lazy).

however, pattern matching is new (I learned some when reading erlang books), I'm still trying to see problems in a pattern matching way.

always fun to learn a new language.

Blog Archive