Saturday, December 03, 2016
notes from aws re:invent 2016:
Amazon Lightsail – The Power of AWS, the Simplicity of a VPS
provides $5 vm same as digital ocean, using shadow vpc, can connect to other services in your vpc via vpc peering. i think they will expend it in the future by providing more integration with aws services. now it's not worth switching from digital ocean.
one of the most useful upgrade, you can actually do dynamic content/decision on cdn edges, that will be very helpful.
Amazon Athena – Interactive SQL Queries for Data in Amazon S3
another very useful feature, you can use kinesis firehose to push data to s3 or cloud watch logs at s3, then run SQL queries directly on them (bucket/prefix/).
AWS Batch – Run Batch Computing Jobs on AWS
poor man's EMR. jobs are run at containers, in a more simple and small scale comparing to EMR. actually I think it will be more useful for me than EMR.
Blox – New Open Source Scheduler for Amazon EC2 Container Service
not a service but an open source project, might only be useful if you use ecs.
Amazon Pinpoint – Hit your Targets with AWS
I think you'll need to integrate your mobile app to use their mobile analytics service,
AWS Shield – Protect your Applications from DDoS Attacks
use cloudflare instead I guess.
AWS CodeBuild – Fully Managed Build Service
quite a few new services are run on containers, this is one of them. I think not useful if you already have a jenkins.
Amazon Polly – Text to Speech in 47 Voices and 24 Languages
Amazon Lex – Build Conversational Voice & Text Interfaces
these two machine learning service is quite good in my opinion.
time to see what everyone else is using: What is your technology stack?
I'm building a simple app with daveray/seesaw: Seesaw turns the Horror of Swing into a friendly, well-documented, Clojure library, I don't see any popular library for javafx yet, so I stick with seesaw.
the most useful tutorial I found is this one (PDF): Seesaw, GUI
it introduces me seesaw.dev
namespace, which includes show-options
and show-events
for an component, very useful.
created a lein
template for aws lambda, put it on usrjim/lambda-template
it allows me create a lambda function very easily, with scripts to create/update on aws and get logs from cloudwatch
Wednesday, December 07, 2016
this is a very cool talk on clojure/conj 2016: Proto REPL, a New Clojure Development and Visualization Tool
I tried it, but still hard to switch to it from emacs
will use it when I need to do debugging or charts
about debugging, this is a tool for clojure: Sayid – Debug and Profile Clojure
of course Rich Hickey's Spec-ulation Keynote is a must watch.
no Semantic Versioning, pretty much like the way golang does. (but seems people do like versioning)
here another article about dealing with new api / old api: Codebase Refactoring (with help from Go)
working with Semantic UI recently, pretty cool.
the documentation is excellent.
I love jq, there's another tool for working with json: simeji/jid: json incremental digger
more useful than jq
in some ways.
however, it doesn't support wildcard, no way to filter properties from an array. someone did a clone and use jq
for filtering: fiatjaf/jiq: jid on jq, nice.
Thursday, December 08, 2016
TIL Message Broker for AWS IoT is the only aws service that supports websocket.
for mqtt, there is a clojure client for it: Machine Head, Clojure MQTT Client
working on setting up a single sign-on service: Using SAML to set up federated SSO
this article How to Set Up Federated Single Sign-On to AWS Using Google Apps provides a good overview of how it works.
also this one: SAML Tutorial: How SAML Authentication Works
onelogin also has a test app for quick setup: PHP SAML Authentication Examples
clojure developer Raynes (Anthony Grimes) has passed away. I used his clojail, lazybot, conch, fs and the super useful Try Clojure
Thursday, December 15, 2016
packt offers 10 days free trial on all their books: Mapt, their daily free book will return on 2017.
and all their titles are $5 now, I will give a quick look on as many as possible books, buy it to read later if it's good.
I'm browsing DevOps for Web Development, learned jenkins pipeline, need to try it.
also browsing Mastering Akka
Clojure Concurrency is not available yet, Clojure Programming Cookbook looks good, it covers some interesting libraies.
while browsing AWS Administration – The Definitive Guide, I knew two new tools for stress test:
DevOps for Networking is good, totally worth it.
learned librarian-puppet from Extending Puppet - Second Edition
I can define modules in a Puppetfile
, then use librarian-puppet install
to deploy it.
it solves many problems of using puppet
:
- I don't need to use
r10k
- I don't need to deploy all modules in a masterless setup
- can specify git branch/tag/commit
- It supports path in a git repo, only a directory will be fetched
- can use tarballs
working with jwt using php, the library I used is lcobucci/jwt, it's simple and easy to use.
another more powerful library is Spomky-Labs/jose, which supports jwe, but I couldn't install it via composer
, it also more complicated.
jwt is not for sensitive data, a token can be invalid, but its data can still be extracted, use jwe if you need encryption.
use jti for one time token, usually generate a uuid as jti
if you set the iat
claim, make sure server time are in sync.
TIL The Virtual Table Mechanism Of SQLite, and Exposing Redis as a Virtual Table
my last project used leveldb for a key-value storage, it works really well. I kinda prefer file-based database now.
I'm reading The Joy of Clojure, slowly.
I learned there is array-map, if you need a sorted map.
because sorted-map-by or sorted-map may eat your elements when return from comparing.
the implementation of a pos
function is fascinating, it's still a long long road before I can reach that kind of elegance.
Saturday, December 17, 2016
setup a deployment with Deployer, like Capistrano, but in PHP
it's easier to maintain, however, documentation is not enough. I need to check source code in order to understand some tasks.
it still gets the job done, so it's good.
candid82/joker: Small interpreted dialect of Clojure written in Go, it would be so cool when I can use chan
and go
the repl is still buggy, reader errors happen quite often.
kibit and eastwood help you to do static analysis on your clojure codes
Friday, December 30, 2016
I found more and more web pages couldn't be converted by my readability client, it's time for an upgrade.
I'm a mozilla user, its reader view works on many pages. I decided to switch to their library.
here's a quick test, clone the mozilla/readability repo, and test with:
const url = require('url');
const jsdom = require('jsdom');
const Readability = require('./index').Readability;
const target = 'https://medium.com/the-vue-point/vue-in-2016-8df71d98bfb3';
const ut = url.parse(target);
jsdom.env({
url: target,
done: function (err, window) {
var article = new Readability(ut, window.document).parse();
console.log(article.title);
console.log(article.byline);
console.log(article.content);
}
});
reading PostgreSQL Server Programming
I worked with some stored procedure in Oracle before - moving them to application side. also mysql doesn't support it, so I never consider to do functions inside database before.
but this book convinced me it actually make sense to do functions inside database, and I'm happy that I read this book.
Are You Load Balancing Wrong? - ACM Queue is a good article, I used load balancer for resilience, and didn't think about capacity at all.
monitoring is important, no matter which purpose of the load balancer is.
Saturday, December 31, 2016
when I checked jsdom, I was so impressed by their example:
// Print all of the news items on Hacker News
var jsdom = require("jsdom");
jsdom.env({
url: "http://news.ycombinator.com/",
scripts: ["http://code.jquery.com/jquery.js"],
done: function (err, window) {
var $ = window.$;
console.log("HN Links");
$("td.title:not(:last) a").each(function() {
console.log(" -", $(this).text());
});
}
});
jquery selector is the best, it's so easy to extract content from html using jsdom
I remember using spring boot to launch a ssh server, but I don't want the spring framework.
Apache MINA is a framework which can be used to build a ssh server.
I'm reading their User Guide, hope I could find something useful there.
reading of this year:
|---------+--------------------------------------------------------------------------|
| 2016-01 | The Varnish Book |
| | Mastering Linux Network Administration |
| | Amazon Web Services in Action |
| | Practical Linux Topics |
|---------+--------------------------------------------------------------------------|
| 2016-02 | The Go Programming Language |
| | Clojure Applied |
| | Advanced UNIX Programming |
|---------+--------------------------------------------------------------------------|
| 2016-03 | Clojure Reactive Programming |
| | Practical Packet Analysis |
|---------+--------------------------------------------------------------------------|
| 2016-04 | The Art of Monitoring |
| | Mastering Clojure |
|---------+--------------------------------------------------------------------------|
| 2016-05 | Advanced Programming in the UNIX Environment |
| | Clojure for Java Developers |
| | Release It!: Design and Deploy Production-Ready Software |
| | Kotlin Language Reference |
| | Java Network Programming |
| | TCP/IP Sockets in Java, Second Edition |
|---------+--------------------------------------------------------------------------|
| 2016-06 | Neural networks and deep learning |
| | Puppet for Containerization |
| | Java Network Programming |
| | Learn You Some Erlang for Great Good! |
|---------+--------------------------------------------------------------------------|
| 2016-07 | Learn You Some Erlang for Great Good! |
|---------+--------------------------------------------------------------------------|
| 2016-08 | Hacking with React |
| | Go in Action |
|---------+--------------------------------------------------------------------------|
| 2016-09 | Go in Practice |
| | Programming in Lua |
|---------+--------------------------------------------------------------------------|
| 2016-10 | Web Development with Go: Building Scalable Web Apps and RESTful Services |
| | Programming in Lua |
| | ØMQ - The Guide |
| | PHP 7 Programming Blueprints |
| | Pro Spark Streaming |
|---------+--------------------------------------------------------------------------|
| 2016-11 | Modern Cljs: A series of tutorials on ClojureScript |
| | Mastering Clojure |
| | ØMQ - The Guide |
| | Linux Journal |
| | Professional Clojure |
|---------+--------------------------------------------------------------------------|
| 2016-12 | The Joy of Clojure |
| | PostgreSQL Server Programming |
|---------+--------------------------------------------------------------------------|
that's the end of my 2016. happy new year!
Blog Archive
- Newer Entries
- 2017 January
- 2017 February
- 2017 March
- 2017 April
- 2017 May
- 2017 June
- 2017 July
- 2017 August
- 2017 September
- 2017 October
- 2017 November
- 2017 December
- 2018 January
- 2018 February
- 2018 March
- 2018 April
- 2018 May
- 2018 June
- 2018 July
- 2018 August
- 2018 September
- 2018 October
- 2018 November
- 2018 December
- 2019 January
- 2019 February
- 2019 March
- 2019 April
- 2019 May
- 2019 July
- 2019 October
- 2019 November
- 2019 December
- 2020 August
- 2020 September
- 2020 October
- 2020 November
- 2020 December
- 2021 January
- 2021 February
- 2021 March
- 2021 April
- 2021 May
- 2021 June
- 2021 August
- 2021 September
- 2021 December
- 2022 March
- 2022 April
- 2022 May
- 2022 June
- 2022 July
- 2022 August
- 2022 September
- 2022 October
- 2022 November
- 2022 December
- 2023 January
- 2023 February
- 2023 March
- 2023 April
- 2023 July
- 2023 August
- 2023 September
- 2023 October
- 2023 November
- 2023 December
- 2024 January
- 2024 February
- 2024 March
- 2024 April
- 2024 May
- 2024 June
- 2024 August
- 2024 September
- Older Entries
- 2016 November
- 2016 October
- 2016 September
- 2016 August
- 2016 July
- 2016 June
- 2016 May
- 2016 April
- 2016 March
- 2016 February
- 2016 January
- 2015 December
- 2015 November
- 2015 October
- 2015 September
- 2015 August
- 2015 July
- 2015 June
- 2015 May
- 2015 April
- 2015 March
- 2015 February
- 2015 January
- 2014 December
- 2014 November
- 2014 October
- 2014 September
- 2014 August
- 2014 March
- 2014 February
- 2014 January
- 2013 December
- 2013 October
- 2013 July
- 2013 June
- 2013 May
- 2013 March
- 2013 February
- 2013 January
- 2012 December
- 2012 November
- 2012 October
- 2012 September
- 2012 August