7/28/2016

Make Mac xcode settings better

Preferences -> Text Editing -> Check numbers
Preferences -> Text Editing -> Check Page guide at column: 80
Preferences -> Text Editing -> Indent width: 2
Preferences -> Fonts and Colors -> Shift to select them all, change the font to Consolas, Size 13

Make Linux screen command settings better

==== Copy to ~/.screenrc
defscrollback 99999
altscreen on
vbell on
vbell_msg ''
termcapinfo *  vb=:

7/22/2016

Java commonly used util functions

Jave:
=====
Paths.get(a, b);  // path join
String.format("%s, haha ", a);

Dependency Injection: Guice
https://github.com/google/guice/wiki/Motivation

https://github.com/google/guice/wiki/GettingStarted

Java docs:
==========

http://www.grpc.io/grpc-java/javadoc/io/grpc/stub/StreamObserver.html

Server:
=======
Apache Struts
Apache Tiles

Jetty as the servelet container

Shell/Bash frequently used commands

Bash Commands:
==============
# Find all cpp files with debug in it.
$find . -name \*.cpp -exec grep -q "debug" '{}' \; -print

# Import Bash script tips
http://www.davidpashley.com/articles/writing-robust-shell-scripts/

if [ -z "${ZONE-}" ]  # if ZONE is null
if [ -n "${ZONE-}" ]  # if ZONE is non-zero
if [ -f "/cluster/env.sh" ] then  # if file exists
  echo "haha"
fi

RSRC=${1:-"/var/www"}  # default to /var/www if the first arg is null.

# Get group members
getent group docker


status (an init conf file in /etc/init/*.conf)

# Rename file extensions
for i in `ls --color=never`; do j=`echo $i | awk -F "." '{print $1}'`; echo $j; mv $i $j.bin; done
# Rename a file with leading 0 prefix
for i in `ls *.jpg`; do j=`echo $i | awk -F "." '{print $1}'`; k=$(printf "%06d" $j); mv $i $k.jpg; done

# To find and replace
sed -i 's/haha/xixi/g' $input
use -i to change the input file directly


for i in `grep -r "2009 Google" * -l`; do sed -i 's/2009\ Google/2010\ Google/g' $i; done
for i in `find . -name BUILD`; do echo $i; sed -i 's/\"\/\/third_party/\"@deepmap_base\/\/third_party/g' $i; done
for i in `find . -name BUILD`; do for j in common point_cloud protos third_party public anga_clients; do sed -i "s/\"\/\/$j/\"@deepmap_base\/\/$j/g" $i; rm -rf $j; done; buildifier $i; done
for i in `find . -iname "SCon*"`; do if grep -q "\\.a" $i; then echo $i; fi done

# Loop a bash array
ids=(1 2 3)

for i in ${ids[@]} do echo $i; done;

for i in {0..10}; do echo $i; done;

# Loop file content: each line
cat run.txt | while read line; do echo $line; done

# Change file name
for i in `cat files.txt`; do echo $i; j=`echo $i | awk -F hgt '{print $1}'`; mv $i ${j}.hgt.zip; done

# Change files names based on pages
j=0
for i in `ls *.pdf`; do mv $i "第${j}页.pdf"; j=$(($j+1)); done

# Sort by size
du -sh * | gsort -h

# cd to real path of a link
cd -P

# List files with full absolute path
ls -d -1 $PWD/*

# tar filter
find . -name '*.php' | tar -cvjf my.tar.bz2 --files-from -

http://geolite.maxmind.com/download/geoip/database/GeoLite2-Country.tar.gz -O - | tar -xz
The wget option -O specifies a file to which the documents is written, and here we use -, meaning it will written to standard output and piped to tar and the tar flag -x enables extraction of archive files and -z decompresses, compressed archive files created by gzip.

Kubernetes frequently used commands

Kubernetes:
===========
# Start cluster
# install virtualbox. Enable VT-x and disable secure boot from BIOS if applicable. Then install minikube and kubectl
minikube start
minikube dashboard # To access the dashboard
# Check namespace
minikube get namespace
# Delete a terminating namespace
NS=`kubectl get ns |grep Terminating | awk 'NR==1 {print $1}'` && kubectl get namespace "$NS" -o json   | tr -d "\n" | sed "s/\"finalizers\": \[[^]]\+\]/\"finalizers\": []/"   | kubectl replace --raw /api/v1/namespaces/$NS/finalize -f -
# Get logs
minikube logs --file=logs.txt

Failed? BE CAREFUL, ALL SERVICES WILL BE GONE. Try:
$minikube stop
$minikube delete && rm -rf ~/.minikube && rm -rf ~/.kube

k cluster-info
kubectl create ns minio-dev # create namespace
k describe pod [pod name]
k get nodes --show-labels
kubectl cordon [node-name]  # Prevent a node from scheduling jobs
kubectl uncordon [node-name] # Resume scheduling jobs on node
k get pod [pod name] -o go-template={{.status.podIP}}
k get pods
k get pods -n [minio-dev]  # -n means namespace
k delete pod [pod name]
k delete pod [pod_name] -n [namespace]
k delete -n default pod [your-pod-name]
k run hello-node --image=hello:v2 --port=8088
k expose pod [pod name] --port 8088
k scale deployment hello-node --replicas=4
k edit deployment hello-node
k apply -f [?]/a.yaml
k get deployment my-nginx -o yaml
k logs [pod id]
k get nodes --show-labels
k port-forward pod/minio 9000 9090 -n minio-dev # Forward port
k port-forward svc/minio -n minio-dev 9090  # Forward port
k rollout restart deployment [deployment_name] -n [namespace]
# Get bash access
kubectl exec -it [pod name, e.g. minio-766dbb8d6d-b6fcq] -n minio-dev  -- /bin/bash

#Careful, service will be down
kubectl replace -f docs/user-guide/nginx/nginx-deployment.yaml --force

http://kubernetes.io/docs/user-guide/docker-cli-to-kubectl/

Kubernetes cluster is running.  The master is running at:

  https://52.34.246.212


The user name and password to use is located in /home/liangzou/.kube/config

7/20/2016

Docker commonly used commands

Docker:
=======
d info
d inspect [container id]
d ps -a
d images
# Fetch image
d pull "image"
d rm [container id]
d rmi [image id]
d run -it # interactive tty
      --rm # remove after exiting
      -p 8888:8080 publish container port 8080 to 8888. To access: localhost:8888
      -v /home/liangzou:/home/liangzou # mounts the host drive to container
      --mac-address 70:85:c2:53:5d:4e # Make the mac address the passed in value.
d commit [container id] # create a new image from the existing docker
# d commit a5ab749ad736 harbor.prod.deepmap.ai/dazel/core-builddeps-dazel:ubuntu1804
d history # history of an image
d run -it --rm tomcat /bin/bash  # to check the container content
d attach --sig-proxy=false [container]
d search ubuntu # search for images
d tag [image id] name:tag # d tag 80ff9c14b9e0 core-builddeps-dazel:ubuntu1804
d exec # runs a new command in a running container.
d kill [container id]
# Delete all stopped containers
d rm `d ps -aq`

# Run a docker first
d run -it -d -v /var/lib/jenkins/workspace/:/root/user --mac-address 70:85:c2:53:5d:4e  776923679937.dkr.ecr.us-west-2.amazonaws.com/buildenv:latest
d exec `d ps -f ancestor=776923679937.dkr.ecr.us-west-2.amazonaws.com/buildenv:latest -q`  bash -c "cd /root/user/DeepMap-Presubmit && bazel build --verbose_failures --spawn_strategy=standalone --genrule_strategy=standalone  -- //experimental/..."
d cp [containerid]:/file/path /host/path

# To detach the tty without exiting the shell, use the escape sequence Ctrl+p + Ctrl+q

# Why does docker attach hang?
https://stackoverflow.com/questions/35573698/why-does-docker-attach-hang

If instead of enter you would start typing a command, you would not see the extra empty prompt line. If you were to run

$ docker exec -it --user root <container-ID-or-name> bash 

7/19/2016

How to use meld for git and configure meld diff tool colors

sudo apt install meld

emacs ~/.gitconfig
Add:
[diff]
    tool = meld
[merge]

    tool = meld

=====
Configure meld external editor:
emacs -fh -geometry 180x160 {file}

=====
For Version: 3.0+
sudo emacs /usr/share/meld/meld.css
For syntax colors:
sudo emacs /usr/share/gtksourceview-3.0/styles/classic.xml

---
Change classic.xml magenta definition to

7/18/2016

Make emacs settings better

For Aquamacs:
The way to change starting window (a.k.a frame) position and size:
Click Preferences | search for default-frame-alist, add var left 300, width 125, height 60, top 0 | Click the state and save for future
Do the same for initial-frame-alist

Command to have a better default GUI:
emacs -fh -geometry 180x160 {file}

sudo apt install xclip
wget http://www.emacswiki.org/emacs-en/download/xclip.el /home/$USER/.emacs.d/xclip.el

==== (Copy to ~/.emacs)
;; Added by Package.el.  This must come before configurations of
;; installed packages.  Don't delete this line.  If you don't want it,
;; just comment it out by adding a semicolon to the start of the line.
;; You may delete these explanatory comments.

(package-initialize)

(add-to-list 'default-frame-alist '(height . 80))

(add-to-list 'default-frame-alist '(width . 180))

;;(require 'py-yapf)

;;(add-hook 'python-mode-hook 'py-yapf-enable-on-save)
;;(add-hook 'prog-mode-hook 'highlight-numbers-mode)

(setq-default indent-tabs-mode nil)
(setq default-tab-width 4)
(setq column-number-mode t)
(add-to-list 'auto-mode-alist '("BUILD" . python-mode))
(set-background-color "#F5E7D1")
(custom-set-variables
 ;; custom-set-variables was added by Custom.
 ;; If you edit it by hand, you could mess it up, so be careful.   
 ;; Your init file should contain only one such instance.
 ;; If there is more than one, they won't work right.
 '(blink-cursor-mode nil)
 '(column-number-mode t)
 '(inhibit-startup-screen t)
 '(package-selected-packages
   (quote
    (py-yapf yaml-mode clang-format fill-column-indicator)))
 '(py-yapf-options
   (quote ("--style" "{based_on_style: google}"))))
(custom-set-faces
 ;; custom-set-faces was added by Custom.
 ;; If you edit it by hand, you could mess it up, so be careful.
 ;; Your init file should contain only one such instance.
 ;; If there is more than one, they won't work right.
  ;; https://github.com/cypro666/yahei.consolas-font to download and install YaHei Consolas
 ;; '(default ((t (:family "YaHei Consolas Hybrid" :slant normal :weight normal :height 105 :width normal :foundry "outline"))))
 ;; Ubuntu system
 ;; DejaVu Sans Mono
 ;; Hack if the font is installed
 '(default ((t (:family "DejaVu Sans Mono" :foundry "PfEd" :slant normal :weight normal :height 90 :width normal))))
 '(c-annotation-face ((t (:inherit font-lock-constant-face :foreground "yellow"))))
 '(cursor ((t (:background "red"))))
 '(font-lock-comment-delimiter-face ((t (:inherit font-lock-comment-face :foreground "forest green"))))
 '(font-lock-comment-face ((t (:foreground "forest green"))))
 '(font-lock-constant-face ((t (:foreground "#A52A2A"))))
 '(font-lock-doc-face ((t (:foreground "forest green"))))
 '(font-lock-function-name-face ((t (:foreground "black"))))
 '(font-lock-keyword-face ((t (:foreground "royal blue" :weight bold))))
 '(font-lock-string-face ((t (:foreground "#A52A2A"))))
 '(font-lock-type-face ((t (:foreground "medium blue" :weight bold))))
 '(font-lock-variable-name-face ((t (:foreground "black"))))
 '(font-lock-builtin-face ((t (:foreground "#8817a2"))))
 '(isearch ((t (:background "#ff7f00" :foreground "white"))))
 '(lazy-highlight ((t (:background "#00F" :foreground "#FFF"))))
 '(show-paren-match ((t (:background "dark gray" :foreground "blue")))))
(set-face-attribute 'region nil :background "#ff7f00" :foreground "#fff")
(put 'upcase-region 'disabled nil)
(setq linum-format "%03d ")
(setq backup-directory-alist
      `((".*" . ,temporary-file-directory)))
(setq auto-save-file-name-transforms
      `((".*" ,temporary-file-directory t)))
(global-linum-mode t)
;; disable auto indent in c/c++
(setq-default c-electric-flag nil)
(setq-default c-syntactic-indentation nil)
;; Bottom status bar
;;(setq-default mode-line-format nil)
(setq-default python-indent-offset 4)
(setq-default python-indent 4)
(add-hook 'before-save-hook 'delete-trailing-whitespace)
(electric-indent-mode -1)
(global-set-key (kbd "RET") 'newline-and-indent)
;; Windows postion
(setq initial-frame-alist '((left . 650) (top . 0)))
(when window-system
  (set-frame-size (selected-frame) 115 80))
(if (functionp 'tool-bar-mode) (tool-bar-mode -1))
;; better scrolling
(setq mouse-wheel-scroll-amount '(4 ((shift) . 8))) ;; one line at a time
(setq mouse-wheel-progressive-speed nil) ;; don't accelerate scrolling
(setq mouse-wheel-follow-mouse 't) ;; scroll window under mouse
(setq scroll-step 1) ;; keyboard scroll one line at a time
(setq scroll-conservatively 10000)

(setq auto-window-vscroll nil)
(require 'dired)
(defun dired-mouse-find-file (event)
  "In Dired, visit the file or directory name you click on."
  (interactive "e")
  (let (window pos file)
    (save-excursion
      (setq window (posn-window (event-end event))
            pos (posn-point (event-end event)))
      (if (not (windowp window))
          (error "No file chosen"))
      (set-buffer (window-buffer window))
      (goto-char pos)
      (setq file (dired-get-file-for-visit)))
    (if (file-directory-p file)
        (or (and (cdr dired-subdir-alist)
                 (dired-goto-subdir file))
            (progn
              (select-window window)
              (dired file)))
      (select-window window)
      (find-file-other-window (file-name-sans-versions file t)))))
(define-key dired-mode-map [mouse-2] 'dired-mouse-find-file)
(setq split-height-threshold nil)

(setq split-width-threshold 160)

;;Matching parenthesis
(require 'paren)
(show-paren-mode 1)
(set-face-attribute 'show-paren-match nil :weight 'extra-bold)
(setq ring-bell-function 'ignore)
; Ctrl+right   => forward word
(global-set-key "\M-[1;5C"    'forward-word)
; Ctrl+left    => backward word
(global-set-key "\M-[1;5D"    'backward-word) 
(global-set-key (kbd "C-x k") 'kill-this-buffer)
(define-key input-decode-map "\e[1;5A" [C-up])

(define-key input-decode-map "\e[1;5B" [C-down])

;; load emacs 24's package system. Add MELPA repository.
(when (>= emacs-major-version 24)
  (require 'package)
  (add-to-list
   'package-archives
   ;; '("melpa" . "http://stable.melpa.org/packages/") ; many packages won't show if using stable
   ;;'("melpa" . "http://melpa.milkbox.net/packages/")
   ;; In China, use this
   '("melpha" . "http://mirrors.tuna.tsinghua.edu.cn/elpa/melpa/")

   t))
(defalias 'yes-or-no-p 'y-or-n-p)
(setq frame-title-format
    (list (format "%s %%S: %%j " (system-name))

    '(buffer-file-name "%f" (dired-directory dired-directory "%b"))))

;; Column ruler, edge
;;(require 'fill-column-indicator)
(setq-default fill-column 80)
;;(define-globalized-minor-mode global-fci-mode fci-mode (lambda () (fci-mode 1)))
;;(global-fci-mode 1)
(setq fci-handle-truncate-lines nil)
(setq fci-rule-color "#606060")
;; Column ruler end

;; (require 'clang-format)
;; (defun clang-format-buffer-smart ()
;;   "Enable clang when the mode is C++"
;;   (when (eq major-mode 'c++-mode)
;;     (clang-format-buffer))
;;   (when (eq major-mode 'c-mode)
;;     (clang-format-buffer)))


;; (add-hook 'before-save-hook 'buildifier)
;; (add-hook 'before-save-hook 'clang-format-buffer-smart)

;; (defcustom buildifier-bin "buildifier"
;;   "Location of the buildifier binary."
;;   :type 'string
;;   :group 'buildifier)

;; (defcustom buildifier-path-regex
;;   "BUILD\\|WORKSPACE"
;;   "Regular expression describing base paths that need buildifier."
;;   :type 'string
;;   :group 'buildifier)

;; (defun buildifier ()
;;   "Run buildifier on current buffer."
;;   (interactive)
;;   (when (and (string-match buildifier-path-regex
;;                            (file-name-nondirectory
;;                             (buffer-file-name)))
;;              (executable-find buildifier-bin))
;;     (let ((p (point))
;;           (tmp (make-temp-file "buildifier")))
;;       (write-region nil nil tmp)
;;       (let ((result (with-temp-buffer
;;                       (cons (call-process buildifier-bin tmp t nil)
;;                             (buffer-string)))))
;;         (if (= (car result) 0)
;;             (save-excursion
;;               (erase-buffer)
;;               (insert (cdr result)))
;;           (warn "%s failed: %s" buildifier-bin (cdr result)))
;;         (goto-char p)
;;         (delete-file tmp nil)))))
(global-set-key (kbd "C-x p") (lambda () (interactive) (other-window -1))) 

Don't use rm in Linux. Use trash command

Why?
It happened to me twice. I used rm -rf in a directory which is not what I wanted to purge. It was a disaster.

Install trash-cli and don't use rm -rf anymore.

7/15/2016

How to fix sougou pinyin after fcitx crashes or fcitx has high CPU usage

If fcitx has high CPU usage, kill it first.

In a terminal run:
fcitx &

Then give it sometime, like 45 seconds or 1 minute for sougyou pinyin to go back to normal. That's it!

Thank YOU, Eclipse!!

Every time I did something stupid to purge my whole folder, it's Eclipse that saved my work.
I really want to say THANK YOU Eclipse! The best code editor ever!

In memory MySQL for java | Embedded MySQL for java unit testing

Mainly used the idea from:
http://blog.palominolabs.com/2011/10/03/embedded-mysql-on-java-with-connectormxj-and-64-bit-linux/
And
https://github.com/airlift/testing-mysql-server

The idea is to use mysql-connector-mxj, but the development has been discontinued. Therefore the newest version that came with it is Mysql 5.5.9. That's included in mysql-connector-mxj-db-files.

That's too old and I want to use a newer version: 5.7.5.

The goal here is therefore to build the new mysql-connector-mxj-db-files jar file to include the new Mysql version.

Here is what I did:

1. Download the newer version MySQL to: ~/Downloads/mysql-5-7-5/mysql-5.7.5-m15-linux-glibc2.5-x86_64

2. Run the command:
bin/mysql_install_db --insecure  --user=mysql --datadir=/tmp/dbfiles/5-7-5/data --lc-messages-dir=./share/

cd /tmp/dbfiles/5-7-5/data
jar -cf ../data_dir.jar *

This step generates the data folder to /tmp/dbfiles/5-7-5/data. Then I create the data_dir.jar file. The data_dir.jar file should be included in the mysql-connector-mxj-db-files jar.

Another command to run:
jar -cf /tmp/dbfiles/5-7-5/share_dir.jar share/

The share_dir.jar should also be included in the mysql-connector-mxj-db-files jar.

3. Create the mysql-connector-mxj-db-files.jar with newer MySQL version.

Unzip the original mysql-connector-mxj-db-files.jar file. Copy the mysql, mysqld file, data_dir.jar, share_dir.jar to appropriate locations.

Run another jar -cf bla bla command to create the final mysql-connector-mxj-db-files.jar.

7/13/2016

MySQL commonly used commands

Remote access:
In order to connect remotely you have to have MySQL bind port: 3306 to your machines IP in my.cnf. Then you have to have created the user in both localhost and '%' wildcard and grant permissions on all DB's as such . See below:
my.cnf
#Replace xxx with your IP Address 
bind-address        = xxx.xxx.xxx.xxx
then
CREATE USER 'myuser'@'localhost' IDENTIFIED BY 'mypass';
CREATE USER 'myuser'@'%' IDENTIFIED BY 'mypass';
Then
GRANT ALL ON *.* TO 'myuser'@'localhost';
GRANT ALL ON *.* TO 'myuser'@'%';

Depending on your OS you may have to open port 3306 to allow remote connections.
Specific to Amazon EC2:

  • The bind-address should be: 0.0.0.0
  • Your machine ip address should be added to amazon security group.


How To Get The Current Epoch Time (Unix Timestamp)

Perl time
PHP time()
Ruby Time.now (or Time.new). To display the epoch: Time.now.to_i
Python import time first, then int(time.time())
Java long epoch = System.currentTimeMillis()/1000;
Microsoft .NET C# epoch = (DateTime.Now.ToUniversalTime().Ticks - 621355968000000000) / 10000000;
VBScript/ASP DateDiff("s", "01/01/1970 00:00:00", Now())
Erlang calendar:datetime_to_gregorian_seconds(calendar:now_to_universal_time( now()))-719528*24*3600.
OR
element(1, now()) * 10000 + element(2, now()).
MySQL SELECT unix_timestamp(now())
PostgreSQL SELECT extract(epoch FROM now());
Oracle PL/SQL SELECT (SYSDATE - TO_DATE('01-01-1970 00:00:00', 'DD-MM-YYYY HH24:MI:SS')) *
24 * 60 * 60 FROM DUAL
SQL Server SELECT DATEDIFF(s, '1970-01-01 00:00:00', GETUTCDATE())
JavaScript Math.round(new Date().getTime()/1000.0) getTime() returns time in milliseconds.
Unix/Linux Shell date +%s
PowerShell Get-Date -UFormat "%s" Produces: 1279152364.63599
Actionscript (new Date()).time
Other OS’s Command line: perl -e "print time" (If Perl is installed on your system)
ColdFusion (CFML) MX 6.1+ #int( getTickCount() / 1000 )#
Bash Command Line: date +%s