jchk - /blog/2022-06

Wednesday, June 08, 2022

under tmux suddenly I found the mouse is not working, changed tmux.conf many times still didn't work

turned out the problem is the terminal app, Allow Mouse Reporting (cmd + r) is off, means it won`t report mouse events

Sunsetting Atom, we all knew it when microsoft bought gitlab

I was still using sublime even when atom came out, not much memory about it

last couple days I've been working on tuning gitlab ci pipeline speed, some notes

cache or artifacts or kaniko cache or base image

cache

as mentioned in How cache is different from artifacts, generally speaking, share cache between branches, artifacts between stages

our projects are using yarn, so use yarn.lock as cache key

job:
  script:
    - echo 'yarn-offline-mirror ".yarn-cache/"' >> .yarnrc
    - echo 'yarn-offline-mirror-pruning true' >> .yarnrc
    - yarn install --frozen-lockfile --no-progress
  cache:
    key:
      files:
        - yarn.lock
    paths:
      - .yarn-cache/

remember to use --frozen-lockfile

the default cache:policy pull-push downloads the cache when job starts and then upload it when job finishes

review the policy when you have parallel jobs

also use policy: pull if you don't need to upload the cache

for clean install, use cache: [] to disable cache

cache stores in runner, usually should use runner that tagged with cache label

artifacts

simuliar to cache, artifacts are attached to jobs, by default jobs in later states will download artifacts from previous jobs

so if there're previous jobs defined artifacts and you don't need it, remember to set dependencies: [] to skip downloads

kaniko cache

I found using kaniko cache is easier for maintaining the ci yaml, we don't need to think about carrying cache/artifacts between stages/jobs

we just need to know, if docker layer is not changed, cache should be applied

for yarn install caching, instead of

COPY . .
RUN yarn install --frozen-lockfile

COPY package.json yarn.lock .
RUN yarn install --frozen-lockfile
COPY . .

so the layer cache "key" will be package.json and yarn.lock files, and able to be reuse for other builds

also remember to clear cache to shorten the snapshot time

RUN yarn cache clean

and exclude .git folder, add to .dockerignore:

.git*
.git/**/*

and lastly, do not add --cache=true in your job if you don't need to keep the cache, to avoid the snapshot time

overall it is a bit slower than using cache / artifacts, but I found it much easier to maintain

base image

for common commands/tools (eslint for example), try build them into a base image

(arguments below are for kaniko/executor)

you can also use images that built in previous stages (by accepting a --build-arg):

ARG BASE_IMAGE

FROM $BASE_IMAGE as base
COPY --from=base /path /target

put ARG above FROM to make it global, useful for multiple stage builds

use --target and --skip-unused-stages to target a special build for local docker-compose

some other minor things:

if everything you need is in the container, you don't need to checkout source files, add this to skip it:

variables:
  GIT_STRATEGY: none

skip the whole job if not required, for example, only run the job when files changed in project folder

only:
  changes:
    - project/**/*

use only: !reference [previous-job, only] in later jobs to avoid copy/paste to too many places

use needs to start a job in next stage even some jobs in current stage is still running

only trigger/skip the job by commit message:

rules: 
  - if: $CI_COMMIT_MESSAGE =~ /\[build storybook]/

there is Merge request pipelines, only triggers when commit from a merge request

I didn't find it useful because we don't have many self-created branches

only:
  - merge_requests

do not run pipeline for new branch (usually a new merge requests created from gitlab ui)

workflow:
  rules:
    - if: $CI_COMMIT_BEFORE_SHA != "0000000000000000000000000000000000000000"

also only/except will be replaced by rules, but I still prefer using only/except

Friday, June 10, 2022

good documentation for modern identity solution: Identity Management (Ory Kratos) Concepts

Sunday, June 12, 2022

this is so cool: Useful utilities and toys over DNS

Lightning Memory-Mapped Database, mentioned in this comment

node wrapper: Venemo/node-lmdb: Node.js binding for lmdb

this is another one, support deno: kriszyp/lmdb-js: Simple, efficient, ultra-fast, scalable data store wrapper for LMDB

looks like could replace redis for caching

to know more about lmdb: [Howard Chu - LMDB [The Databaseology Lectures - CMU Fall 2015]](https://www.youtube.com/watch?v=tEa5sAh-kVk)

CMU Database Group has many interesting talks

to me team autonomy probably the key to efficiency: How Big Tech Runs Tech Projects and the Curious Absence of Scrum

reading Modern Systems Programming with Scala Native

Friday, June 17, 2022

learned something new from 4 macOS Screenshot Tricks To Impress Your Co-Workers

so if I don't want to save any files:

cmd + shift + 4, then hit space
ctrl + option + click (copy to clipboard)
open Preview (using cmd + space), cmd + n to paste then follow by cmd + shift + a (annotate)
once done select the area and copy it
paste to slack

redbean 2.0 release notes

redbean is amazing, it just a zip and you can add lua scripts/html files to it and then run it

and you can find some interesting implementation built on top of it:

Sunday, June 19, 2022

I have a set of simple tools for terminal, but sometimes I also want to access them in spotlight, so maybe wrap them as an app

I tried a little bit of Wails, but seems too much work

then I found Fluid – Turn any webpage into a real Mac App is perfect for my need

there's another simuliar app: Flotato - lost in a sea of browser tabs?

project like pojala/electrino: Desktop runtime for apps built on web technologies, using the system's own web browser engine is also interesting, but fluidapp is good enough for me

Monday, June 20, 2022

working on rules, using these libaraies:

the online demo: Rule Editor

Wednesday, June 22, 2022

Reproducible Development with Devcontainers, presentation here

so it's a .devcontainer folder comes with the project, and start a container (with sleep infinity) for command utils/profiles/vscode setup etc.

it's not a full development environment, but a common shell for developers, very nice

it could solve lots of versioning/environment variables/testing issues, must try

Blog Archive