Jim Cheung

Friday, December 04, 2015

podcasts to listen:

PHP 7.0.0 Released, I'm quite happy with it, even though it becomes more java-like. but I believe more static-type-like features make PHP a better language. some php7 articles to read:

after read this article, Perl 6 Is Fun, I wanna play some perl again.

reading December 2015 Issue of Linux Journal, using google authentication to guard your sshd with two-factor auth one is very useful and will definitely implement it to my servers.

finished Clojure for the Brave and True, it's the best book I've read in 2015. I'll put my notes on here soon.

now reading The Joy of Clojure, Second Edition, slowly.

Saturday, December 05, 2015

finally rewrote the homepage app with clojure, development is quite smooth, deployment is not:

I used compojure.route/resources to serve my public files, but lein ring uberjar will pack files under resources into the jar, I don't want that.

then I changed to compojure.route/files, but kept getting file not found error. turned out ring-defaults was actually the one who handled public files. once I removed site-defaults public files are working fine.

when deployed to the server, I put it in a jre docker container, ran with a simple java -jar command. but again all files were not found. the problem is the relative path is based on where the java command started, I have to wrap it with a shell script, cd to the app folder and ran the jar file, then everything works fine.

the response is not as fast as using nodejs, I may do some test and switch to Undertow instead of jetty.

anyways, very good experience. I enjoy working with clojure. next step is port my news site to clojure as well.

Sunday, December 06, 2015

felt a bit laggy after changed to clojure, I suspected it's because of jetty. so I want to try undertow, the simplest way is changed to Immutant, it can "eat" a compojure and that's it:

(defn -main []
  (web/run app))

app here is a normal compojure (defroutes app ...)

got another issue when deployed: immutant.web/run default binds to localhost, didn't notice that until got connection problem with docker port forwarding.

but it does feel smoother after switched to undertow.

Monday, December 07, 2015

about aws elastic load balancing:

resolve an elb end point you'll get a dynamic ip, putting varnish/nginx/haproxy in front of an elb can be quite tricky. for performance reason, these kind of application will cache the resolved dns result, they will be broken when elb ip changed.

elb cname's ttl time is 60 seconds, may keep connection as long as 1 hour. I think that's one of the reason why elb doesn't support http/2 yet, its short ttl and dynamic nature doesn't work well with persistent connection (websocket, server push).

I didn't find an elegant solution for this problem yet, keep searching.

Tuesday, December 08, 2015

on ubuntu,

Wednesday, December 09, 2015

setting up google authenticator guided with Linux Journal's article, however, the guide doesn't work well.

it said update /etc/pam.d/sshd with following:

# add this line to top
auth required pam_google_authenticator.so

# comment out this line
@include common-auth

the purpose is to avoid server still asks for password even the verification code is passed. but it will block all users without google authenticator.

some online articles will use:

auth sufficient pam_google_authenticator.so nullok

not preferable as well, because this allows user skips google authenticator and use password login instead.

the correct way is mentioned in this article: Securing SSH with TOTP:

auth [success=done new_authtok_reqd=done default=die] pam_google_authenticator.so nullok

sufficient is an alias of [success=done new_authtok_reqd=done default=ignore], so a little bit modification here by changing it with default=die.

nullok allows users without google authenticator setup to login with normal way (public key).

for vagrant some article suggests put this line to /etc/pam.d/sshd:

auth [success=1 default=ignore] pam_succeed_if.so user=vagrant

however it doesn't work for me and actually it's not necessary if vagrant user doesn't setup a google authenticator.

so the final version is:

  1. update /etc/pam.d/sshd, add to the top:
    auth [success=done new_authtok_reqd=done default=die] pam_google_authenticator.so nullok
  2. update /etc/ssh/sshd_config, change or add:
    PubkeyAuthentication yes
ChallengeResponseAuthentication yes
PasswordAuthentication yes
AuthenticationMethods publickey,keyboard-interactive:pam
    about the QR, secret key and emergency scratch codes:
  3. you can setup a valid authenticator in multiple devices if you copied the QR and secret key. but it's very insecure.
  4. you need to copy emergency scratch codes to a safe place. they're one-time verification codes, once logged in should re-run google-authenticator to setup a new secret key (which will also trash the old lost secret key)

test it with vagrant and always keep one root user ssh session open, it's quite dangerous and will lock your server easily with wrong config.

Wednesday, December 16, 2015

(It's not a good idea committing sensitive data to your repository), hiera-eyaml allows you put encrypted data to hiera.

using gpg to create a password protected file from a ascii text source:

gpg -c -a private.txt

there're so many podcasts to listen, especially Software Engineering Daily, they have so many interesting people and topic everyday. just a few good picks recently:

I love Immutant 2, I can run multiple apps on it with different ports or virtual-hosts, really cool. I'd like to try it's messaging and caching libries too.

Tuesday, December 22, 2015

learn a trick from here: How to cycle through reverse-i-search in BASH?

tag a command and can search that tag later:

command #useful

ctrl+r #useful

set hostname under ubuntu:

sudo hostnamectl set-hostname some.example.com

unlike sudo hostname some.example.com, this command will also update /etc/hostname so that setting remains even after reboot.

Wednesday, December 23, 2015

Vagrant 1.8 released. linked clones and snapshot both look quite useful.

I didn't try Otto yet, but definitely will do.

Amethyst a tiling window manager for os x, much nicer than apps I used before: isnap, better touch, etc..

Sunday, December 27, 2015

this christmas I stayed away from computer and just be with my family.

5 AWS mistakes you should avoid and discussion on HN. But as someone also mentioned, I don't like cloudformation, I rather do it by sdk, it feels more controllable and manageable than cloudformation.

having a little bit trouble with my dnsmasq, found a nice way Debugging nameservers using dig +norec.

Perl 6 is finally here. first thing i wanna do in 2016 is learning perl 6.

Perl 6 Introduction is a good place to get started.

listened couple of podcasts about clojurescript (both are David Nolen), kinda interested in it and want to try it.

some recent news about clojurescript:

a very quick start from this comment on ClojureScript Year in Review | Hacker News:

$ lein new reagent myapp
$ cd myapp
$ lein figwheel

# app will be available at localhost:3449
# edit src/cljs/myapp/core.cljs and try it out

Tuesday, December 29, 2015

got a problem with capistrano deployment, the public key ssh authentication failed, it kept asking for a password.

the reason is when I enabled google authenticator, keyboard-interactive authentication method was added. the net-ssh ruby gem is not working well with it (probably because it's an old version).

I don't want to take away google authenticator because of this, the solution is using the Match User sshd_config(5) to disable keyboard-interactive for the deploy user.

to debug capistrano ssh connections:

ssh_options[:verbose] = :debug

reading Nginx High Performance, it is quite good.

when I read to the tcp_nodelay directive, I remembered this post: how we shaved 200ms off every POST request, and the real John Nagle comment on it

Wednesday, December 30, 2015

read this from proggit: Springer have made a bunch of maths books available for free, here are the direct links

I can't read maths books, but found few good ones:

TLDR pages is a funny project, but man -k keyword (or apropos keyword) with less is quite enough.

Deckatron is a presentation platform written by clojure.

resolved another weird problem of varnish returning empty cache. the problem is varnish was making HTTP/1.0 requests to backend (nginx), something is wrong with that. change to HTTP/1.1 solved the issue:

sub vcl_backend_fetch {
  set bereq.proto = "HTTP/1.1";
}

update: I think this article has a good explanation on what's is wrong with the HTTP/1.0 backend requests:

the article also provides examples on how to check it, very useful. (I think I'll read all the blog posts under their varnish category.

Thursday, December 31, 2015

end of 2015. the plan on 2016 will be clojure/clojurescript and perl6, and spend more time with family.

happy new year!

finally here's what I've been read/reading in 2015 (from grep "reading" blog/2015-*):

|---------+---------------------------------------------------------------------------|
| 2015-01 | Filthy Rich Clients                                                       |
|         | Pro Spring Batch                                                          |
|---------+---------------------------------------------------------------------------|
| 2015-02 | Reactive Web Applications with Play                                       |
|         | Play for Java                                                             |
|         | Practical Load Balancing                                                  |
|         | Java 8 in Action                                                          |
|---------+---------------------------------------------------------------------------|
| 2015-03 | Beginning Spring                                                          |
|---------+---------------------------------------------------------------------------|
| 2015-04 |                                                                           |
|---------+---------------------------------------------------------------------------|
| 2015-05 | Living Clojure                                                            |
|         | Web Development with Clojure                                              |
|         | Clojure Programming (3rd attempt)                                         |
|         | Lisp Hackers                                                              |
|         | lots of Linux Journal                                                     |
|---------+---------------------------------------------------------------------------|
| 2015-06 | Computing for Ordinary Mortals                                            |
|         | Linux Journal                                                             |
|         | Seven Concurrency Models in Seven Weeks                                   |
|---------+---------------------------------------------------------------------------|
| 2015-07 | 21st Century C                                                            |
|         | Big Data: Principles and best practices of scalable realtime data systems |
|         | Effective Ruby: 48 Specific Ways to Write Better Ruby                     |
|         | Clojure Web Development Essentials                                        |
|---------+---------------------------------------------------------------------------|
| 2015-08 | Clojure Programming (4th attempt)                                         |
|         | Big Data: Principles and best practices of scalable realtime data systems |
|---------+---------------------------------------------------------------------------|
| 2015-09 | DevOps Troubleshooting: Linux Server Best Practices                       |
|---------+---------------------------------------------------------------------------|
| 2015-10 | PostgreSQL: Up and Running                                                |
|         | Java SE8 for the Really Impatient                                         |
|         | The Joy of Clojure                                                        |
|         | Nginx Essentials                                                          |
|---------+---------------------------------------------------------------------------|
| 2015-11 | Clojure for the Brave and True                                            |
|         | The Performance of Open Source Applications                               |
|---------+---------------------------------------------------------------------------|
| 2015-12 | The Joy of Clojure, Second Edition                                        |
|         | Nginx High Performance                                                    |
|---------+---------------------------------------------------------------------------|
Blog Archive