Jim Cheung

Saturday, April 01, 2017

my elasticsearch dies a lot recently, when I heard Algolia has free plan (you'll need to put their logo on your site), I decided to give it a try.

hacker news readers will know, algolia is the search engine they're using.

algolia supports large number of languages and frameworks, no clojure though.

it's simple to use java client with clojure

add the dependency [com.algolia/algoliasearch "2.8.0"]

the :import looks like this:

(ns algolia.core
  (:require [clojure.data.json :as json])
  (:import (com.algolia.search ApacheAPIClientBuilder)
           (com.fasterxml.jackson.databind ObjectMapper)))

to instantiate the ApacheAPIClient object:

(defn init-client
  [app-id api-key]
  (-> (ApacheAPIClientBuilder. app-id api-key)
      (.build)))

instantiate Index object:

(defn init-index
  [client index-name]
  (.initIndex client index-name))

and sync a map to algolia:

(let [client (init-client app-id api-key)
      index (init-index client "test_articles")
      data (-> (json/write-str {:objectID 1
                                :title "foo"
                                :body "bar"})
               (ObjectMapper.)
               (.readTree s))]
  (.add-object index data))

here, data is a com.fasterxml.jackson.databind.JsonNode converted from a clojure map.

the java api client has async version as well.

the search part is even easier, algolia's web explorer is very useful, it displays raw queries and results of your search, just put that into your code.

I tried use stuartsierra/component: Managed lifecycle of stateful objects in Clojure before, found it's complicated.

now my codebase becomes larger, I read it again and I found it useful.

there is also tolitius/mount: managing Clojure and ClojureScript app state since (reset) for the same purpose.

continue watching clojure west 2017 talks:

Why Clojure? - Derek Slager

he has some good opnions on microservices.

I also have a single repo for most clojure projects. the tool mentioned in the video could help me: lein-monolith

Faster Delivery with Pedestal and Vase - Paul deGrandis

in the talk he talked about Pedestal, all things are interceptor and composed together.

interceptor pattern is not a new idea, can ben found in Pattern-Oriented Software Architecture Volume 2: Patterns for Concurrent and Networked Objects and Volume 4: A Pattern Language for Distributed Computing

also talked about vase, Paul introduced vase in one episode of cognitect podcast

vase is on top of pedestal, and solve data problem in a microservice way.

$ lein new vase project-name

Fearless JVM Lambdas - John Chapin

serverless, java and aws lambda

Wednesday, April 12, 2017

when I create a new luminus project, I noticed there is a Capstanfile, looks like this (removed comments):

base: cloudius/osv-openjdk8
cmdline: /java.so -jar /my-project/app.jar
build: lein uberjar
files:
  /my-project/app.jar: ./target/uberjar/my-project.jar

it's for the operation system named: OSv

looks like a lightweight way to run vm, never heard about it before, but I'm definitely interested.

ProgresTouch RETRO TINY keyboard from Tokyo. pretty good.

I also have an ikbc poker II keyboard, beautiful but need some time to get used to it.

I have trouble with original fn + wasd as arrows setup, I finally decided to try the programming feature of it:

first, caps lock is useless to me, while fn key is heavily used, I want to map caps lock as fn.

it can be done by turning on dip 1 (caps <-> left win) and 3 (left win <-> fn)

but then I will lost my left win key, which is quite useful under os x (as command)

I found the solution online and the trick is, BEFORE turn on dip switches, map the caps to left win first by programming:

  1. fn + right ctrl to enter programming mode
  2. tap caps
  3. tap left win
  4. tap pn to complete
  5. fn + right ctrl to exit programming mode

then disconnect keyboard, turn on dip switches 1 and 3, re-connect keyboard. now caps should work as fn and pn + left win works as left win

typing pn every time is nonsense, fn + right shift will switch to program layer, you don't need to hold pn for programmed keys (hold pn will access normal layer keys while you're on program layer)

other custom keys I set:

| fn + j, k, h, l | as arrows                                |
| fn + esc        | as ~                                     |
| fn + v, b       | as ctrl + left, right (switch workspace) |

once I have a comfortable set of arrow keys, I'm quite productive on the 60% keyboard.

my next target is a 40% keyboard (no number row), looking forward to its arrival.

Professional Clojure has one chapter for datomic, a nice introduction.

datomic is just like clojure, it changes the way you think.

Thursday, April 13, 2017

some emacs stuffs:

first, a comprehensive OrgMode tutorial

second, learned restclient.el package after watched Emacs Rocks! Episode 15: restclient-mode

one cool feature didn't mention in the video:

C-c C-u: copy query under the cursor as a curl command

when I got json response, I want to hide/show some blocks, there is a built-in feature in emacs for this: Hideshow

M-x hs-minor-mode

| C-c @ C-h | Hide the current block |
| C-c @ C-s | Show the current block |
| C-c @ C-c | Toggle current block   |

Saturday, April 15, 2017

zbarimg can help you extract urls from images contain qr code.

some history books added to my read list:

Saturday, April 22, 2017

things I read about clojure last couple days:

core.async

On the Judicious Use of core.async

nullable

Robust Clojure: The best way to handle nil

to summarize: treat nil as type (concept of Maybe or Nullable) instead of value.

use these functions/libraries to help you deal with nil:

if-some:

(if-some [it (get {:a 1 :b 2} :c)]
  (+ 1 it)
  nil)

some-> and some->>:

(some->> {:a 1 :b 3} :c vector (filter even?) (map inc) first) ;;=> nil

fnil:

(def safe-inc (fnil inc 0))

(safe-inc (get {:a 1 :b 2} :c)) ;=> 1

core.match:

(match (foo) ;; pretend `foo` is function that returns a map
  nil (log "foo failed")
  {:ms t :user u :data data} (do (log "User: " u " took " t " seconds.")
                                 data))

Cats: Category Theory and algebraic abstractions for Clojure.:

(require '[cats.core :as m])
(require '[cats.monad.either :as either])

@(m/mlet [x (if-let [v (foo)]
              (either/right v)
              (either/left))

          y (if-let [v (bar x)]
              (either/right v)
              (either/left))

          z (if-let [v (goo x y)]
              (either/right v)
              (either/left))]

  (m/return (qux x y z)))

get and or to handle default values:

(-> (get {:a 1 :b 2} :c 0) inc) ;=> 1

(-> :c
    {:a 1 :b 2}
    (or 41)
    inc) ;=> 42

further readings:

spec

stathissideris/spec-provider: Infer clojure specs from sample data. Inspired by F#'s type providers.

native app

JUXT Blog: Writing ClojureScript native apps is easy

an interesting suggestion: Pattern Matching for Java

I wish golang has pattern matching, to make their if err != nil to something like elixir's:

with {:ok, result1} <- square(a),
     {:ok, result2} <- half(result1), 
     {:ok, result3} <- triple(result2),
     {:ok, result4} <- do_even_more_stuff(result3)
do
  IO.puts “Success! the result is #{result4}”
else 
  {:error, error} -> IO.puts “Oops, something went wrong: #{error}”
end

btw, rust supports pattern matching:

fn main() {
    let file_name = "foobar.rs";
    match find(file_name, '.') {
        None => println!("No file extension found."),
        Some(i) => println!("File extension: {}", &file_name[i+1..]),
    }
}

Tout est Terrible is a good read.

I like the comment Doing It Right costs Lots Of Money. and Ninety-ninety rule

Lots of good resources inside: Ask HN: Which companies have the best blogs written by their engineering team?

docker now has a upper level project: moby:

Introducing Moby Project: a new open-source project to advance the software containerization movement

the relationship is like fedora(moby) and rhel(docker),

I'm not sure it's a good idea for docker, but for developers don't like docker (like me), it is good that people can build their solutions using smaller components:

however, I think we'll wait couple more years for common practices about how we should do containers.

found a useful tool for viewing logs: lnav: The Log File Navigator

just apt-get install lnav to install. I'll write some notes later.

Tmux and Vim - even better together, my colleague suggested using them before, but I don't quite want to change my workflow for now.

there is a much more useful article mentioned in the article: Vim Splits - Move Faster and More Naturally

put in .vimrc:

set splitbelow
set splitright

nnoremap <C-J> <C-W><C-J>
nnoremap <C-K> <C-W><C-K>
nnoremap <C-L> <C-W><C-L>
nnoremap <C-H> <C-W><C-H>

useful shortcuts:

"Max out the height of the current split
ctrl + w _

"Max out the width of the current split
ctrl + w |

"Normalize all split sizes, which is very handy when resizing terminal
ctrl + w =

"Swap top/bottom or left/right split
Ctrl+W R

"Break out current window into a new tabview
Ctrl+W T

"Close every window in the current tabview but the current one
Ctrl+W o

last month I wrote about vim text objects, also should read documentation about object motions and selections

Saturday, April 29, 2017

need to remove some sensitive data from git repo, found this tool is helpful: BFG Repo-Cleaner by rtyley

it is very easy to use (compares to git filter-branch), simply:

$ bfg -D api_key.conf repo.git

few notes:

about agile:

Slow Is Fast

the quote from Rich Hickey:

“sprinters” only run short distances and “Agile” solves this by firing the starting gun every 100 meters.

is from his famouse talk Simple Made Easy (just search "Sprinter")

I worked for a team which runs scrum, then a no-agile team, now in agile again.

I have to say I was more productive and happy under the no-agile environment. I can spend my time to tackle complex problem and it usually produce more high quality codes and pretty fast after that.

I made bad trade-offs/decisions when I need to do daily standup, because it is always very hard to estimate how much time needs to complete a small part of the system.

agile doesn't make estimation easier or more accurate, it encourages programmers make bad trade-offs and fake promises.

gRPC-Web: Moving past REST+JSON towards type-safe Web APIs

rest, graphql or grpc, I still don't have an answer for now.

Instead of containerization, give me strong config & deployment primitives

just read the discussion on hackernews, you'll know how messy the devops world is. no one agrees on anything (some probably agree on kubernetes but i'm not convinced yet)

I found two good articles from skyliner:

and they wrote the Fourteen Months with Clojure article.

working on a project to pipeline analytics data to some storage backend.

there're few ga alternatives:

piwik is the most popular once, the problem with piwik is its backend is MySQL, probably needs more effort to maintain it for large scale.

people will take its JavaScript Tracking Client and drop its MySQL backend.

snowplow has few collectors, but we want to do serverless.

aws api gateway can transform requests (enrich):

and POST to another services, like kinesis firehose:

which the storage endpoint is S3, and it supports encryption (through KMS) and compression (gzip only if needs read from redshift)

that's actually what we wanna do, but no server setup and maintenance.

if mapping template is not enough for the enrichment, we can still pipe data to aws lambda function and push it to kinesis stream

you still have the flexibility to solve quite complex logics by composing multiple lambda and kinesis stream, and there is tool to push kinesis stream to kinesis firehose:

kinesis firehose also support binding a lambda function to transform data:

Amazon Kinesis Firehose Data Transformation

further reading:

Wednesday, May 03, 2017

a really cool emacs package: rbanffy/selectric-mode: Make your Emacs sound like a proper typewriter.

I'm trying to use emacs as much as possible:

winner-mode is very useful, it registers window layout autometically, and use c-c left and c-c right to switch.

I'll c-x 1 to maximize the working window, and c-c left to un-maximize.

file management, dired for sure:

s to toggle different sorts + to create directory m to mark file u to unmark U to unmark all g to refresh ^ goes up one level < and > to move to next/previous sub-directory j to quickly jump to a file

to simulate midnight commander under emacs, first set this variable:

(setq dired-dwim-target t)

then in separate dired mode window, press C will default copy to target directory in other window.

to select multiple files, use % m regex

dired supports zip files, however, it can not extract file from the zip (open in buffer only)

Blog Archive