Jim Cheung

Saturday, December 03, 2016

notes from aws re:invent 2016:

Amazon Lightsail – The Power of AWS, the Simplicity of a VPS

provides $5 vm same as digital ocean, using shadow vpc, can connect to other services in your vpc via vpc peering. i think they will expend it in the future by providing more integration with aws services. now it's not worth switching from digital ocean.

Lambda@Edge

one of the most useful upgrade, you can actually do dynamic content/decision on cdn edges, that will be very helpful.

Amazon Athena – Interactive SQL Queries for Data in Amazon S3

another very useful feature, you can use kinesis firehose to push data to s3 or cloud watch logs at s3, then run SQL queries directly on them (bucket/prefix/).

AWS Batch – Run Batch Computing Jobs on AWS

poor man's EMR. jobs are run at containers, in a more simple and small scale comparing to EMR. actually I think it will be more useful for me than EMR.

Blox – New Open Source Scheduler for Amazon EC2 Container Service

not a service but an open source project, might only be useful if you use ecs.

Amazon Pinpoint – Hit your Targets with AWS

I think you'll need to integrate your mobile app to use their mobile analytics service,

AWS Shield – Protect your Applications from DDoS Attacks

use cloudflare instead I guess.

AWS CodeBuild – Fully Managed Build Service

quite a few new services are run on containers, this is one of them. I think not useful if you already have a jenkins.

Amazon Polly – Text to Speech in 47 Voices and 24 Languages

Amazon Lex – Build Conversational Voice & Text Interfaces

these two machine learning service is quite good in my opinion.

time to see what everyone else is using: What is your technology stack?

I'm building a simple app with daveray/seesaw: Seesaw turns the Horror of Swing into a friendly, well-documented, Clojure library, I don't see any popular library for javafx yet, so I stick with seesaw.

the most useful tutorial I found is this one (PDF): Seesaw, GUI

it introduces me seesaw.dev namespace, which includes show-options and show-events for an component, very useful.

created a lein template for aws lambda, put it on usrjim/lambda-template

it allows me create a lambda function very easily, with scripts to create/update on aws and get logs from cloudwatch

Wednesday, December 07, 2016

this is a very cool talk on clojure/conj 2016: Proto REPL, a New Clojure Development and Visualization Tool

I tried it, but still hard to switch to it from emacs

will use it when I need to do debugging or charts

about debugging, this is a tool for clojure: Sayid – Debug and Profile Clojure

of course Rich Hickey's Spec-ulation Keynote is a must watch.

no Semantic Versioning, pretty much like the way golang does. (but seems people do like versioning)

here another article about dealing with new api / old api: Codebase Refactoring (with help from Go)

working with Semantic UI recently, pretty cool.

the documentation is excellent.

I love jq, there's another tool for working with json: simeji/jid: json incremental digger

more useful than jq in some ways.

however, it doesn't support wildcard, no way to filter properties from an array. someone did a clone and use jq for filtering: fiatjaf/jiq: jid on jq, nice.

Thursday, December 08, 2016

TIL Message Broker for AWS IoT is the only aws service that supports websocket.

for mqtt, there is a clojure client for it: Machine Head, Clojure MQTT Client

working on setting up a single sign-on service: Using SAML to set up federated SSO

this article How to Set Up Federated Single Sign-On to AWS Using Google Apps provides a good overview of how it works.

also this one: SAML Tutorial: How SAML Authentication Works

onelogin also has a test app for quick setup: PHP SAML Authentication Examples

clojure developer Raynes (Anthony Grimes) has passed away. I used his clojail, lazybot, conch, fs and the super useful Try Clojure

Thursday, December 15, 2016

packt offers 10 days free trial on all their books: Mapt, their daily free book will return on 2017.

and all their titles are $5 now, I will give a quick look on as many as possible books, buy it to read later if it's good.

I'm browsing DevOps for Web Development, learned jenkins pipeline, need to try it.

also browsing Mastering Akka

Clojure Concurrency is not available yet, Clojure Programming Cookbook looks good, it covers some interesting libraies.

while browsing AWS Administration – The Definitive Guide, I knew two new tools for stress test:

DevOps for Networking is good, totally worth it.

learned librarian-puppet from Extending Puppet - Second Edition

I can define modules in a Puppetfile, then use librarian-puppet install to deploy it.

it solves many problems of using puppet:

working with jwt using php, the library I used is lcobucci/jwt, it's simple and easy to use.

another more powerful library is Spomky-Labs/jose, which supports jwe, but I couldn't install it via composer, it also more complicated.

jwt is not for sensitive data, a token can be invalid, but its data can still be extracted, use jwe if you need encryption.

use jti for one time token, usually generate a uuid as jti

if you set the iat claim, make sure server time are in sync.

TIL The Virtual Table Mechanism Of SQLite, and Exposing Redis as a Virtual Table

also The CSV Virtual Table

my last project used leveldb for a key-value storage, it works really well. I kinda prefer file-based database now.

I'm reading The Joy of Clojure, slowly.

I learned there is array-map, if you need a sorted map.

because sorted-map-by or sorted-map may eat your elements when return from comparing.

the implementation of a pos function is fascinating, it's still a long long road before I can reach that kind of elegance.

Saturday, December 17, 2016

setup a deployment with Deployer, like Capistrano, but in PHP

it's easier to maintain, however, documentation is not enough. I need to check source code in order to understand some tasks.

it still gets the job done, so it's good.

candid82/joker: Small interpreted dialect of Clojure written in Go, it would be so cool when I can use chan and go

the repl is still buggy, reader errors happen quite often.

kibit and eastwood help you to do static analysis on your clojure codes

Friday, December 30, 2016

I found more and more web pages couldn't be converted by my readability client, it's time for an upgrade.

I'm a mozilla user, its reader view works on many pages. I decided to switch to their library.

here's a quick test, clone the mozilla/readability repo, and test with:

const url = require('url');
const jsdom = require('jsdom');
const Readability = require('./index').Readability;

const target = 'https://medium.com/the-vue-point/vue-in-2016-8df71d98bfb3';
const ut = url.parse(target);

jsdom.env({
    url: target,
    done: function (err, window) {
        var article = new Readability(ut, window.document).parse();
        console.log(article.title);
        console.log(article.byline);
        console.log(article.content);
    }
});

reading PostgreSQL Server Programming

I worked with some stored procedure in Oracle before - moving them to application side. also mysql doesn't support it, so I never consider to do functions inside database before.

but this book convinced me it actually make sense to do functions inside database, and I'm happy that I read this book.

Are You Load Balancing Wrong? - ACM Queue is a good article, I used load balancer for resilience, and didn't think about capacity at all.

monitoring is important, no matter which purpose of the load balancer is.

Saturday, December 31, 2016

when I checked jsdom, I was so impressed by their example:

// Print all of the news items on Hacker News
var jsdom = require("jsdom");

jsdom.env({
  url: "http://news.ycombinator.com/",
  scripts: ["http://code.jquery.com/jquery.js"],
  done: function (err, window) {
    var $ = window.$;
    console.log("HN Links");
    $("td.title:not(:last) a").each(function() {
      console.log(" -", $(this).text());
    });
  }
});

jquery selector is the best, it's so easy to extract content from html using jsdom

I remember using spring boot to launch a ssh server, but I don't want the spring framework.

Apache MINA is a framework which can be used to build a ssh server.

I'm reading their User Guide, hope I could find something useful there.

reading of this year:

|---------+--------------------------------------------------------------------------|
| 2016-01 | The Varnish Book                                                         |
|         | Mastering Linux Network Administration                                   |
|         | Amazon Web Services in Action                                            |
|         | Practical Linux Topics                                                   |
|---------+--------------------------------------------------------------------------|
| 2016-02 | The Go Programming Language                                              |
|         | Clojure Applied                                                          |
|         | Advanced UNIX Programming                                                |
|---------+--------------------------------------------------------------------------|
| 2016-03 | Clojure Reactive Programming                                             |
|         | Practical Packet Analysis                                                |
|---------+--------------------------------------------------------------------------|
| 2016-04 | The Art of Monitoring                                                    |
|         | Mastering Clojure                                                        |
|---------+--------------------------------------------------------------------------|
| 2016-05 | Advanced Programming in the UNIX Environment                             |
|         | Clojure for Java Developers                                              |
|         | Release It!: Design and Deploy Production-Ready Software                 |
|         | Kotlin Language Reference                                                |
|         | Java Network Programming                                                 |
|         | TCP/IP Sockets in Java, Second Edition                                   |
|---------+--------------------------------------------------------------------------|
| 2016-06 | Neural networks and deep learning                                        |
|         | Puppet for Containerization                                              |
|         | Java Network Programming                                                 |
|         | Learn You Some Erlang for Great Good!                                    |
|---------+--------------------------------------------------------------------------|
| 2016-07 | Learn You Some Erlang for Great Good!                                    |
|---------+--------------------------------------------------------------------------|
| 2016-08 | Hacking with React                                                       |
|         | Go in Action                                                             |
|---------+--------------------------------------------------------------------------|
| 2016-09 | Go in Practice                                                           |
|         | Programming in Lua                                                       |
|---------+--------------------------------------------------------------------------|
| 2016-10 | Web Development with Go: Building Scalable Web Apps and RESTful Services |
|         | Programming in Lua                                                       |
|         | ØMQ - The Guide                                                          |
|         | PHP 7 Programming Blueprints                                             |
|         | Pro Spark Streaming                                                      |
|---------+--------------------------------------------------------------------------|
| 2016-11 | Modern Cljs: A series of tutorials on ClojureScript                      |
|         | Mastering Clojure                                                        |
|         | ØMQ - The Guide                                                          |
|         | Linux Journal                                                            |
|         | Professional Clojure                                                     |
|---------+--------------------------------------------------------------------------|
| 2016-12 | The Joy of Clojure                                                       |
|         | PostgreSQL Server Programming                                            |
|---------+--------------------------------------------------------------------------|

that's the end of my 2016. happy new year!

Blog Archive