git中文手册

http://rogerdudler.github.io/git-guide/index.zh.html

Git 入门实战

https://speakerdeck.com/icyleaf/git-ru-men-shi-zhan

开发流程

git-flow 是目前流行且比较成熟的开发流程模式,以下是译文:

http://www.oschina.net/translate/a-successful-git-branching-model

取消最后一次提交

http://stackoverflow.com/questions/927358/how-to-undo-the-last-git-commit

run.sh样本

#!/bin/bash

HADOOP_HOME=/usr/lib/hadoop
HADOOP_STREAMING=/usr/lib/hadoop-0.20-mapreduce

orderfromid='12345'
srcid='semid1'


DATETIME=`date -d last-day +%Y%m%d`
DATETIME='{20131202,20131203,20131204,20131205,20131206,20131207,20131208}'
input='/data/logs/webserver/website/hotel/'$DATETIME
output='/homedir/username/keyword_convertion/'$srcid'/'$DATETIME
mapper_file='./mapper.py'
reducer_file='./reducer.py'

$HADOOP_HOME/bin/hadoop jar $HADOOP_STREAMING/contrib/streaming/hadoop-streaming.jar \
    -D mapred.text.key.partitioner.options=-k1,1 \
    -D map.output.key.field.separator=' ' \
    -D mapred.output.key.comparator.class=org.apache.hadoop.mapred.lib.KeyFieldBasedComparator \
    -input $input \
    -output $output \
    -mapper $mapper_file \
    -file $mapper_file \
    -reducer $reducer_file \
    -file $reducer_file \
    -cmdenv srcid="$srcid" \
    -cmdenv orderfromid="$orderfromid" \
    -numReduceTasks 0

mapper样本

#!/usr/bin/python
import sys
import re
import urllib2
import os
from urlparse import parse_qs,urlparse

orderfromid = os.environ['orderfromid']
srcid = os.environ['srcid']

for line in sys.stdin:
  line = line.strip()
  if line[:1] == '#':
    continue

  try:
    date,time,sitename,computername,s_ip,method,uri_stem,uri_query,port,username,c_ip,cs_version,ua,c_cookies,referer,cs_host,sc_status,sc_substatus,sc_win32_status,sc_bytes,cs_bytes,time_taken = line.split(' ')
  except ValueError:
    continue

  query_pairs = parse_qs(uri_query)

  if 'TableName' in query_pairs:
    TableName = query_pairs['TableName'][0]
    cookies={}
    for cookie in c_cookies.split(';+'):
      kv = cookie.split('=')
      if len(kv) != 2:
        continue
      k,v = kv
      cookies[k]=v

    if 'CookieGuid' not in cookies:
      continue
    CookieGuid = cookies['CookieGuid']
    key = CookieGuid+' '+date+' '+time
    if TableName == 'FlowStatiOrder':
      if 'OrderFrom' not in query_pairs or query_pairs['OrderFrom'][0] != orderfromid:
        continue
      value = "type=o&orderid="+query_pairs['OrderId'][0]
    elif TableName == 'FlowStatiData':
      referer_pairs = urlparse(referer)
      referer_query_pairs = parse_qs(referer_pairs.query)
      if 'srcid' not in referer_query_pairs or 'uuid' not in referer_query_pairs or referer_query_pairs['srcid'][0] != srcid:
        continue
      value = "type=d&uuid="+referer_query_pairs['uuid'][0]
    else:
      continue
  else:
    uri_query_pairs = parse_qs(uri_query)
    if 'srcid' not in uri_query_pairs or 'uuid' not in uri_query_pairs or uri_query_pairs['srcid'][0] != srcid:
      continue
    value = "type=d&uuid="+uri_query_pairs['uuid'][0]
  if 'key' in locals():
    print key+"\t"+value

reducer样本

#!/usr/bin/python
import sys
import re
import urllib2
from urlparse import parse_qs
last_line = None

for line in sys.stdin:
  cookie_date_time,value = line.split("\t")
  cookie,date,time = cookie_date_time.split(" ")
  value_pairs = parse_qs(value)

  if last_line:
    last_cookie_date_time,last_value = last_line.split("\t")
    last_cookie, last_date,last_time = last_cookie_date_time.split(' ')
    last_value_pairs  = parse_qs(last_value)
    if 'uuid' in last_value_pairs:
      last_value_uuid = last_value_pairs['uuid'][0].strip()
    else:
      last_value_uuid = 'xxxxxxxxxxx'

    if 'type' in value_pairs and value_pairs['type'][0].strip() == 'o':
      if last_cookie == cookie:
        print last_value_uuid+"\t"+"1"
      else:
        print last_value_uuid+"\t"+"0"
    else:
      print last_value_uuid+"\t"+"0"

  last_line = line

首先安装 sshfs

brew install sshfs

查看fuse4x的说明

brew info fuse4x-kext

执行其中的两行

sudo /bin/cp -rfX /usr/local/Cellar/fuse4x-kext/0.9.2/Library/Extensions/fuse4x.kext /Library/Extensions
sudo chmod +s /Library/Extensions/fuse4x.kext/Support/load_fuse4x

建立映射关系

sshfs charles@seo:~ share

实现本功能的前提是已经实现了”ssh免密码登陆” 新建或者编辑配置文件: ~/.ssh/config

Host machine1
HostName machine1
Port 22
User username1

Host machine2
HostName machine2
Port 22
User username2

实际使用时:

ssh machine1
ssh machine2

配置文件是/etc/my.cnf(或/etc/my.cnf.d/server.cnf)

首先检查所有数据库机器的配置中,数据库datadir的位置, 保证这个位置的owner是mysql, 即让mysql有读写权限.

然后启动主库mysql服务 ~~~bash sudo service mysql start ~~~

修改配置文件

[mysqld]
skip-networking

查看datadir中的错误日志,如果启动后出现错误说mysql库中的表找不到,可能需要执行一下

mysql_upgrade

在主库中执行,看两个值,mysql-bin编号, master_log_pos, 记录下来 ~~~mysql SHOW MASTER STATUS; ~~~

如果SHOW MASTER STATUS返回空结果, 很可能SHOW BINARY LOGS也返回空结果. 这样的话一般是bin-log没有正常启动. 如果datadir对mysql有写权限. 修改配置文件 ~~~bash [server] log-bin=mysql-bin ~~~

登陆从库的linux机器,修改配置文件.假设三台机器,一主二从,那么server-id分别是1,2,3. 需要保证id是唯一的.

[server]
server-id = 2

到从库中执行 ~~~mysql CHANGE MASTER TO MASTER_HOST=’sem1’, MASTER_PORT=3306, MASTER_USER=’rep’,MASTER_PASSWORD = ‘mysqlrep’, MASTER_LOG_FILE=’mysql-bin.000013’, MASTER_LOG_POS=11187; #填入之前show master status记录的数值

START SLAVE; #开启从库同步模式

SHOW SLAVE status; #查看从库同步状态



修改所有机器的配置,skip_name_resolve,否则会经常看到报警
~~~bash
[mysqld]
skip_name_resolve

常用debug方式 ~~~mysql SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1; #忽略slave上的一条错误 SET GLOBAL expire_logs_days = 30; #binlog只保留30天 ~~~