CentOS7安装gitlab-ce

安装配置相关依赖

在系统防火墙中打开HTTP和SSH。

1
2
3
4
5
6
7
8

sudo yum install curl policycoreutils openssh-server openssh-clients -y
sudo systemctl enable sshd
sudo systemctl start sshd
sudo yum install postfix
sudo systemctl enable postfix
sudo systemctl start postfix
sudo firewall-cmd --permanent --add-service=http
sudo systemctl reload firewalld

添加GitLab包服务器并安装

1
2

curl -sS https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh | sudo bash
sudo yum install gitlab-ce -y

也可以下载软件包然后安装

1
2

curl -LJO https://packages.gitlab.com/gitlab/gitlab-ce/packages/el/7/gitlab-ce-XXX.rpm/download
rpm -i gitlab-ce-XXX.rpm

清华镜像

如果这版本不能下载，可以考虑清华版本，

新建/etc/yum.repos.d/gitlab-ce.repo

1
2
3
4
5

[gitlab-ce]
name=Gitlab CE Repository
baseurl=https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el$releasever/
gpgcheck=0
enabled=1

1
2

sudo yum makecache
sudo yum install gitlab-ce

配置并启动GitLab

1

sudo gitlab-ctl reconfigure

浏览并登陆

此时就可以设置用户名和密码了。

Enjoy!!!

IDE开发环境

网上流传的最好的10个用于开发C/C++的IDE环境。

终端链接 tty

Linux tty命令用于显示终端机连接标准输入设备的文件名称。

在Linux操作系统中，所有外围设备都有其名称与代号，这些名称代号以特殊文件的类型存放于/dev目录下。

比如ttyN就是今天说的设备，而sddN等就是硬盘设备。

你可以执行tty(teletypewriter)指令查询目前使用的终端机的文件名称。

官方定义为：

tty - print the file name of the terminal connected to standard input

使用方法比较简单：

1

$ tty [-s][--help][--version]

对于-s选项就是–silent，–quiet，即屏蔽掉输出，仅仅显示一个退出状态。

默认使用

默认情况下显示当前终端

1
2

$ tty
/dev/pts/4

查询谁在以及在那个tty

在Linux里面输入who可以看到目前登陆的用户，而输出信息包括用户名，tty终端，及登陆的时间信息等等。

1
2
3
4
5
6
7
8
9
10
11
12

$ who
user1      pts/4        2017-04-21 19:58 (xxx.xxx.xxx.xxx)
user1      pts/5        2017-04-07 13:41 (:99)
user1      pts/0        2017-04-08 16:31 (:99)
user1      pts/1        2017-04-08 17:12 (:99)
user1      :0           2017-04-15 15:05 (:0)
user1      pts/2        2017-04-15 15:38 (:0)
user2  	   pts/3        2017-04-16 08:53 (:3)
user2      pts/6        2017-04-16 11:01 (:3)
user2      pts/7        2017-04-16 16:49 (:3)
user3      pts/8        2017-04-21 20:05 (xxx.xxx.xxx.xxx)
user3      pts/9        2017-04-21 20:07 (xxx.xxx.xxx.xxx)

而如前面所说，对于write命令其中有一个参数就是指定ttyN的信息。

迷你计算器 bc

.. note::
北斗南辰日夜移，飞走鸟和兔。
元·王哲《卜算子·叹世迷》

Linux bc命令是一种支持任意精度的交互执行的命令。

bc也是一种支持交互式执行语句的任意精度数的语言。与C语言有一些相似之处。 标准数学库也可以由 通过命令行选项使用。

官方定义为：

bc - An arbitrary precision calculator language

使用方法为：

1

$ bc [ -hlwsqv ] [long-options] [  file ... ]

默认实例

默认进入交互环境，直接可以执行计算

1
2
3
4
5
6
7

$ bc
bc 1.07.1
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006, 2008, 2012-2017 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
12*3
36

输入quit即可退出。

通过管道使用

简单的情况可以使用管道来实现，如下：

1
2

$ echo "3.1415926 * 3" | bc
9.4247778

高级一点

可以通过scale来指定一些精度信息，如下可以保留3为有效精度

1
2

$ echo "scale=3; 2/3" | bc
.666

另外还可以使用一些数学函数，比如：

1
2

$ echo "sqrt(36)" | bc
6

还可以方便地使用ibase进行进制的转换，下面分别是输入为111，对应在2、4、8进制下的输出。

1
2
3
4
5
6

$ echo 'ibase=2;111' | bc 
7
$ echo 'ibase=4;111' | bc
21
$ echo 'ibase=8;111' | bc
73

当然也可以通过obase来指定输入进制，如下将输入的8进制的111，分别转换为2、4、8进制。

1
2
3
4
5
6

$ echo 'ibase=8;obase=2;111' | bc
1001001
$ echo 'ibase=8;obase=4;111' | bc
1021
$ echo 'ibase=8;obase=8;111' | bc
111

CASA安装步骤

下载: https://casa.nrao.edu/casa_obtaining.shtml

安装步骤 ： https://casa.nrao.edu/casadocs/latest/usingcasa/obtaining-and-installing

1
2
3

$ export CASAPATH=/the/path/of/casa/
$ cd $CASAPATH
$ tar zxvf casa-release-x.x.x.tar.gz

把下述内容添加到~/.bashrc，这样就可以自启动了

1

export PATH=$PATH:/usr/local/bin/casa-release-5.4.1-31.el7/bin

对于开启了SELinux的Linux而言，可能需要关闭这个功能
setsebool -P allow_execheap=1

pyplot基础图表函数概述

matplotlib可以绘制各种类型的图，基本的几个如下表所示，示例图见下。

plot 示例图

boxplot 示例图

bar 示例图

barh 示例图

polar 示例图

pie 示例图

pas 示例图

specgram 示例图

cohere 示例图

scatter 示例图

step 示例图

hist 示例图

contour 示例图

vlines 示例图

stem 示例图

plot_date 示例图

Matplotlib 简介

至理名言：A picture is worth a thousand words。

MatplotLib 支持交互和非交互的绘图，并且可以保持图像到PNG、PS和其他类型的文件。

支持的输出格式

支持的后端

渲染器的输出格式为：

python-dateutil - 增强datetime

可以使用的集成环境IDE

• Enthought Python Distribution
• Python(x,y)
• Sage
• Anaconda

开始使用 Matplotlib

三种使用Matplotlib的方式

1. pyplot: 提供与Matlab类似的函数式环境，state-machine interface。通常用于interactive ploting的模式，例如ipython，可立即看到绘图结果。使用方式如下：from matplotlib import pyplot as plt,pyplot中提供的函数接口见：http://matplotlib.org/api/pyplot_api.html#pyplot
2. pylab：结合了pyplot（用于作图）和numpy（用于数学计算）。在本书中被嫌弃。使用方式如下：ipython -pylab
3. 面向对象式的

对于面向对象式的，比如下面的代码：

1
2
3
4
5
6
7
8
9

import matplotlib.pyplot as plt
import numpy as np
x = np.arange(0, 10, 0.1)
y = np.random.randn(len(x))
fig = plt.figure()
ax = fig.add_subplot(111)
l, = plt.plot(x, y)
t = ax.set_title('random numbers')
plt.show()

用Matplotlib绘制第一幅图片

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

import matplotlib.pyplot as plt
import numpy as np

len = 10

x = []
y = []
z = []
xx = range(len)
for i in range(len):
    x.append(np.random.rand())
    y.append(np.random.rand())
    z.append(np.random.rand())

plt.plot(xx,x)
plt.plot(xx,y)
plt.plot(xx,z)

plt.show()

效果图

用Matplotlib绘制添加各种信息的图片

在前面的基础上添加上网格，坐标，坐标轴信息和标题等信息

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

import matplotlib.pyplot as plt
import numpy as np

len = 10

x = []
y = []
z = []
xx = range(len)
for i in range(len):
    x.append(np.random.rand())
    y.append(np.random.rand())
    z.append(np.random.rand())

plt.plot(xx,x)
plt.plot(xx,y)
plt.plot(xx,z)
plt.legend(['legend x','legend y','legend z'],loc='best')

plt.grid(True)
plt.xlabel('Points')
plt.ylabel('Random Value')
plt.title('Test data')

plt.show()

效果图

保存图像到文件

1

plt.savefig('plot.png')

绘图键盘快捷键与功能

进入matplotlib环境

1

$ ipython -pylab

在该命令下，

• ion ： 打开交互模式
• ioff：关闭交互模式
• draw：强制重绘

配置Matplotlib

全局配置文件 /etc/matplotlibrc

在该文件中我们可以看到backend : TkAgg，因为TkAgg不基于任何依赖，所以在大部分平台上都会正常工作，这也是因为python默认使用的就是Tk/Tcl。

几个小技巧

1
2
3
4
5

plt.hold() # 是否保留原来图像

plt.interactive(True) # 使能交互模式

plt.legend # 有几个位置，可以使用`loc`参数

绘图的美化

这一章用来美化绘图咯。

颜色选项

通过加上简单的一个参数就可以控制图片的线条颜色了。

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

import matplotlib.pyplot as plt
import numpy as np

len = 10

x = []
y = []
z = []

xx = range(len)
for i in range(len):
    x.append(np.random.rand())
    y.append(np.random.rand())
    z.append(np.random.rand())

# Adding color parameters
plt.plot(xx,x,'y',label='legend x')
plt.plot(xx,y,'m',label='legend y')
plt.plot(xx,z,'c',label='legend z')
plt.legend()

plt.grid(True)
plt.xlabel('Points')
plt.ylabel('Random Value')
plt.title('Colors')

plt.show()

效果图

支持的颜色列表：

颜色的表示方法除了用缩写，还可以使用颜色名字或者16进制的RGB代码，比如下面的三行代码效果是一样的。

1
2
3

plot(xx,x,'r')
plot(xx,x,'red')
plot(xx,x,'#FF0000')

线条类型

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

len = 10

x = []
y = []
z = []

xx = range(len)
for i in range(len):
    x.append(np.random.rand())
    y.append(np.random.rand())
    z.append(np.random.rand())

# Using different line styles
plt.plot(xx,x,'--',label='x')
plt.plot(xx,y,'-.',label='y')
plt.plot(xx,z,':',label='z')
plt.legend()

plt.grid(True)
plt.xlabel('Points')
plt.ylabel('Random Value')
plt.title('Colors')

plt.show()

效果图

线条类型表

点标记类型

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

len = 10

x = []
y = []
z = []
m = []
n = []

xx = range(len)
for i in range(len):
    x.append(np.random.rand())
    y.append(np.random.rand())
    z.append(np.random.rand())
    m.append(np.random.rand())
    n.append(np.random.rand())

# Using different line styles
plt.plot(xx,x,'o',label='x')
plt.plot(xx,y,'^',label='y')
plt.plot(xx,z,'x',label='z')
plt.plot(xx,m,'s',label='z')
plt.plot(xx,n,'D',label='z')
plt.legend()

plt.grid(True)
plt.xlabel('Points')
plt.ylabel('Random Value')
plt.title('Markers')

plt.show()

效果图

标记点类型

组合标记和线类型

其实颜色、线条类型以及点标记是可以组合使用的。

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

len = 10

x = []
y = []
z = []
m = []

xx = range(len)
for i in range(len):
    x.append(np.random.rand())
    y.append(np.random.rand())
    z.append(np.random.rand())
    m.append(np.random.rand())

# Using different line styles
plt.plot(xx,x,'--o',label='x')
plt.plot(xx,y,'-.^',label='y')
plt.plot(xx,z,'-x',label='z')
plt.plot(xx,m,':s',label='z')
plt.legend()

plt.grid(True)
plt.xlabel('Points')
plt.ylabel('Random Value')
plt.title('Line with Markers')

plt.show()

效果图

使用关键词参数来更好地控制绘图

前面的格式化参数虽然很有用，但是还是有些缺点的，比如，不允许我们分别设置点和线的颜色。
所以这里就需要使用关键词参数了。

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

len = 5

x = []
y = []
z = []
m = []

xx = range(len)
for i in range(len):
    x.append(np.random.rand())
    y.append(np.random.rand())
    z.append(np.random.rand())

plt.plot(xx,x,color='blue',linestyle='dashdot',linewidth=2,marker='o',
         markerfacecolor='red',markeredgecolor='black',label='x')
plt.plot(xx,y,color='green',linestyle='solid',linewidth=3,marker='s',
         markerfacecolor='blue',markeredgecolor='red',label='y')
plt.plot(xx,z,color='cyan',linestyle=':',linewidth=2,marker='^',
         markerfacecolor='green',markeredgecolor='blue',label='z')
plt.legend()

plt.grid(True)
plt.xlabel('Points')
plt.ylabel('Random Value')
plt.title('Keyword')

plt.show()

效果图

其实plot能做出许多美轮美奂的图片，这就取决于你的想象力了。

关键词参数列表

设置x轴与y轴的tick标记

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

len = 5

x = []
y = []
z = []
m = []

xx = range(len)
for i in range(len):
    x.append(np.random.rand())
    y.append(np.random.rand())
    z.append(np.random.rand())

plt.plot(xx,x,color='blue',linestyle='dashdot',linewidth=2,marker='o',
         markerfacecolor='red',markeredgecolor='black',label='x')
plt.plot(xx,y,color='green',linestyle='solid',linewidth=3,marker='s',
         markerfacecolor='blue',markeredgecolor='red',label='y')
plt.plot(xx,z,color='cyan',linestyle=':',linewidth=2,marker='^',
         markerfacecolor='green',markeredgecolor='blue',label='z')
plt.legend()

plt.grid(True)
plt.xlabel('Points')
plt.ylabel('Random Value')
plt.title('Keyword')

plt.xticks(range(len),['a','b','c','d','e'])
plt.yticks(np.arange(0,1,0.05))

plt.show()

效果图

绘图类型

前面说的都是线类型，其实matplotlib可以绘制很多类型，如下图所示。

直方图

直方图 主要用于显示出现的频率值，将某一部分数据集放在一个category里面，这个category称谓
bins 。默认的bins值为10，可以通过参数bins来修改该值，比如下面的源码将其更改为25。

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

len = 1000

y = np.random.randn(len)

plt.hist(y,bins=25,color='red',label='Data')
plt.legend()

plt.grid(True)
plt.xlabel('Points')
plt.ylabel('Random Value')
plt.title('Histogram Charts')

plt.show()

效果图

误差条形图

对于实验值，精确的值通常情况下是不存在的，因为有各种各样的误差值，那么此时误差条形图就来了。

且看如何使用该图。

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

x = np.arange(0,10,0.2)
y = np.exp(-x)
e1 = 0.1 * np.abs(np.random.randn(len(y)))
e2 = 0.15 * np.abs(np.random.randn(len(y)))

plt.errorbar(x,y,xerr=e1,yerr=e2,ecolor='red',fmt='-',label='error')

plt.axis([0,10,-0.5,1.2])

plt.legend()

plt.grid(True)
plt.xlabel('Points')
plt.ylabel('Random Value')
plt.title('Error Bar Charts')

plt.show()

效果图

可以看到上面的误差图是对称的，如果是非对称的，只需要将plot的语句更改为如下所示

1

plt.errorbar(x,y,yerr=[e1,e2],ecolor='red',fmt='-',label='error')

非对称效果图

直方图

直方图主要用于可视化两组或者多组值。

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt

plt.bar([1,2,3,4],[4,6,3,8])

plt.grid(True)
plt.xlabel('Points')
plt.ylabel('Value')
plt.title('Bar Charts')

plt.show()

参数

• width：bar的宽度
• color：bar的颜色
• xerr，yerr：bar的误差
• bottom：bar的底部坐标

效果图

更清晰的直方图

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

dict = {'A':40,'B':70,'C':35,'D':86}

for i,key in enumerate(dict):
    print i,key
    plt.bar(i,dict[key])

# 因为宽度默认为0.8，此处的0.4即为每个bar的中间位置
plt.xticks(np.arange(len(dict))+0.4,dict.keys())
# 只显示有用的值
plt.yticks(dict.values())
plt.xlabel('Points')
plt.ylabel('Value')
plt.title('Clear Bar Charts')

plt.show()

效果图

复杂直方图

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

data1 = 10*np.random.rand(5)
data2 = 10*np.random.rand(5)
data3 = 10*np.random.rand(5)

e2 = 0.5 * np.abs(np.random.randn(len(data2)))

locs = np.arange(1,len(data1)+1)

width = 0.27

plt.bar(locs,data1,width=width)

plt.bar(locs+width,data2,yerr=e2,width=width,color='red')

plt.bar(locs+2*width,data3,width=width,color='green')

plt.xticks(locs + width *1.5,locs)

plt.xlabel('Points')
plt.ylabel('Value')
plt.title('Complex Bar Charts')

plt.show()

效果图

饼状图

饼状图主要用于表述某一部分在整体中占得比例。

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt

x = [25,35,45]

labels = ['Good','Failed','Passed']

plt.pie(x,labels=labels)

plt.title('Pie Charts')

plt.show()

效果图

参数

• explode : 所属的部分是否相对于半径有偏移，也就是突出显示的意思
• colors : 颜色控制
• labels ： 标记
• shadow ： 是否有阴影渲染

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt

x = [5,25,35,45,10]

labels = ['Excellent','Good','Failed','Passed','Bad']

explode = [0.1,0.2,0.0,0.1,0.0]

shadow = [0.0,0.1,0.2,0.1,0.0]

plt.pie(x,labels=labels,explode=explode,shadow=shadow)

plt.title('Pie Charts')

plt.show()

效果图

散点图

散点图主要用于鉴定两组变量的潜在关系，也就是相关关系。

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

x = np.random.randn(1000)
y = np.random.randn(1000)

plt.scatter(x,y)

plt.title('Scatter Plot')

plt.show()

效果图

修饰散点图

可以使用下面的几个参数来修饰散点图：

• s：表示点的大小size
• c：表示点的颜色color
• marker：表示点的样式，可以参考前面关于Marker的描述

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

x = np.random.randn(1000)
y = np.random.randn(1000)

size = 50*np.random.randn(1000)
colors = np.random.rand(1000)

plt.scatter(x,y,s=size,c=colors)

plt.title('Scatter Plot')

plt.show()

效果图

极坐标图

极坐标使用了一个完全不同的坐标系，前面的绘图，我们使用的都是笛卡尔坐标系，分别为x与y轴。
而极坐标使用的是半径与角度（或弧度，Matplotlib默认使用角度）。

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

theta = np.arange(0.,2.,1./180.)*np.pi

# Draw a spiral
plt.polar(3*theta,theta/5)
# Draw a polar rose, a pretty function that resembles a flower
plt.polar(theta,np.cos(8*theta))
# Draw a circular
plt.polar(theta,[1.4]*len(theta))

plt.title('Polar Plot')

plt.show()

效果图

可以通过参数rgrids和thetagrid来控制相关的参数显示，比如下面的代码将绘制一个蝴蝶。

rgrids的参数有下面几个：

• radii：The radial distances at which the grid lines should be drawn.
• labels: The labels to display at radii grid. By default, the values from radii would be used, but if not None, then it has to be of the same length as that of radii.
• angle: The angle at which the labels are displayed (by default it’s 22.5°).

thetagrids的参数有下面几个：

• angles: label的位置
• labels: Specifies the labels to be printed at given angles. If None, then the angles values are used, or else the array must be of the same length of angles.
• frac: The polar axes radius fraction at which we want the label to be drawn (1 is the edge, 1.1 is outside, and 0.9 is inside).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

theta = np.arange(0.,2.,1./180.)*np.pi
r = np.abs(np.sin(5*theta)-2.*np.cos(theta))

plt.polar(theta,r)

plt.thetagrids(range(45,360,90))
plt.rgrids(np.arange(0.2,3.,.7),angle=90)

plt.title('Polar Plot')

plt.show()

效果图

说明

转载请注明作者及其出处

github
Homepage
Homepage
CSDN

图片中的文字、注释与箭头

前面的图片我们很简单地添加了x、y轴和标题的问题。
这里我们学着添加图片内部的文字。

其中text中的x、y为坐标值；
figtext为相对坐标，(0,0)为左下角，(1,1)为左上角；

除了text，用的比较多的应该是annotate，既可以写上文字，也可以标记出来具体的点。

annotate的参数：

• width: The width of the arrow in points
• frac: The fraction of the arrow length occupied by the head
• headwidth: The width of the base of the arrow head in points
• shrink: Moves the tip and the base of the arrow some percent away from
  the annotated point and text, in percentage (so 0.05 is equal to 5%)

annotate已经很好用了，不过我们还可以通过arrow来任意指定箭头的位置、大小和方向。

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

x = np.arange(0,4*np.pi,.01)
y = np.sin(x)

plt.plot(x,y)

plt.text(0.1,-0.05,'Sin(x)')
plt.figtext(0.5,.5,'sin(x) center')
plt.annotate('This point must be really \nmean something',xy=(np.pi/2.,1),xytext=(2.9,0.7),
             arrowprops=dict(facecolor='cyan',shrink=0.05))

plt.title('Text && Annotation')

plt.show()

Another

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt

plt.axis([0,10,0,20])

arrstyles = ['-','->','-[','<-','<->','fancy','simple','wedge']

for i,style in enumerate(arrstyles):
    plt.annotate(style,xytext=(1,2+2*i),xy=(4,1+2*i),arrowprops=dict(arrowstyle=style))

for i,style in enumerate(arrstyles):
    plt.annotate('',xytext=(6,2+2*i),xy=(8,1+2*i),arrowprops=dict(arrowstyle=style))

plt.title('Different Arrows')

plt.show()

效果图

Matplotlib 进阶

使用Matplotlib的几种方式

有三种使用Matplotlib的方法：

• pyplot
• pylab：将Matplotlib和NumPy集合在一起，很像Matlab
• 面向对象的方式：这也是最好的一种方式，因为可以对返回的结果进行全方位的控制

同一个程序用上面的三种方法，效果图如下所示：

源码如下所示：

pyplot

1
2
3
4
5
6
7

import matplotlib.pyplot as plt
import numpy as np
x = np.arange(0, 10, 0.1)
y = np.random.randn(len(x))
plt.plot(x, y)
plt.title('random numbers')
plt.show()

pylab

使用ipython -pylab，输入代码如下：

1
2
3
4
5
6

In [1]: x = arange(0, 10, 0.1)
In [2]: y = randn(len(x))
In [3]: plot(x, y)
Out[3]: [<matplotlib.lines.Line2D object at 0x4284dd0>]
In [4]: title('random numbers')
In [5]: show()

这里需要特别注意的是 ipython -pylab 与from pylab import * 是不一样的。

OO

最后来一个面向对象的，代码如下所示：

1
2
3
4
5
6
7
8
9

import matplotlib.pyplot as plt
import numpy as np
x = np.arange(0, 10, 0.1)
y = np.random.randn(len(x))
fig = plt.figure()
ax = fig.add_subplot(111)
l, = plt.plot(x, y)
t = ax.set_title('random numbers')
plt.show()

对比

从上面的代码来看，pylab模式下的代码最少，pyplot次之，面向对象的代码最多。

正如python之禅所说：”Explicit is better than implicit” ，”Simple is better than complex” 。
在简单的交互模式下，pylab或者pyplot是最好的选择，因为他们隐藏了很多复杂性，但是如果我们需要更深一步的控制，
面向对象的API此时就会派上用场了，因为OO模式会清晰地告诉我们从哪里来，到哪里去。
特别是再将Matplotlib嵌入到GUI的时候更是如此。

Matplotlib的几个对象

Subplots

前面的OO模式的代码，来看看几个概念的解释：

• fig = plt.figure()：figure函数返回一个Figure对象，我们可以在其上增加Axes实例
• ax = fig.add_subplot(121)：add_subplot返回一个Axes实例，我们可以在其上绘制图形

add_subplot的使用有三个参数：

1

fig.add_subplot(numrow,numcols,fignum)

比如下面的代码：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

fig = plt.figure()

x = np.arange(0,10,0.1)
y = np.random.randn(len(x))
ax1 = fig.add_subplot(121)
#l, = plt.plot(x,y)
#l.set_color('red')
ax1.plot(x,y,'rs-')
ax1.set_title('Random Numbers 1')
ax1.set_xlabel('x')
ax1.set_ylabel('y')

x = np.arange(0,10,0.1)
y = np.random.randn(len(x))
ax2 = fig.add_subplot(122)
ax2.plot(x,y)
ax2.set_title('Random Numbers 2')

plt.show()

多个Figure

Matplotlib 不仅提供了在一个Figure里面绘制多个Axes的功能，还能绘制多幅Figure。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
fig1 = plt.figure()
ax1 = fig1.add_subplot(111)
ax1.plot([1, 2, 3], [1, 2, 3]);
fig2 = plt.figure()
ax2 = fig2.add_subplot(111)
ax2.plot([1, 2, 3], [3, 2, 1]);
plt.show()

两个坐标轴

有的时候希望在同一副图像上绘制两个数据集。
特别是对同一个X变量，具有不同的Y值。

这里的技巧在于twinx函数，它会创建第二套坐标，并且与第一套坐标重合，来绘制图形。
不过要注意的是，因为使用了twinx，所有的设置都会重置，包括线条的颜色也会是蓝色，所以为了区分，需要设置颜色。

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

x = np.arange(0.1,np.e,0.01)
y1 = np.exp(-x)
y2 = np.log(x)

fig = plt.figure()
ax1 = fig.add_subplot(111)
ax1.plot(x,y1)
ax1.set_ylabel('Y values for exp(-x)')

ax2 = ax1.twinx()
ax2.plot(x,y2,'r')
ax2.set_xlim([0,np.e])
ax2.set_ylabel('Y values for ln(x)')
ax2.set_xlabel('Same X for both exp(-x) and ln(x)')

plt.show()

效果图

对数坐标轴

对数坐标轴主要针对有对数关系或者数据增长或缩减比较迅速的数据。

其中semilogx/semilogy一样是将plot与set_xscale/set_yscale合并为一个命令。

而loglog是将x/y全部设置为对数坐标。

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

x = np.arange(0.,20,0.01)

fig = plt.figure()

ax1 = fig.add_subplot(311)
y1 = np.exp(-x)
ax1.plot(x,y1)
ax1.grid(True)
ax1.set_yscale('log')
ax1.set_ylabel('log y')

ax2 = fig.add_subplot(312)
y2 = np.cos(np.pi*x)
ax2.semilogx(x,y2)
ax2.set_xlim([0,20])
ax2.grid(True)
ax2.set_ylabel('Log X')

ax3 = fig.add_subplot(313)
y3 = np.exp(x/4.)
ax3.loglog(x,y3,basex=3)
ax3.grid(True)
ax3.set_ylabel('Log x and y')

plt.show()

效果图

共享坐标轴

有些情况下，我们希望不同的Figure共享同一个坐标轴，这样在一个坐标改变时会联动其他坐标同时改变。

这对于金融数据、硬件测试、健康状态监测都是很有帮助的。

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib.pyplot as plt
import numpy as np

x = np.arange(20)

fig = plt.figure()

ax1 = fig.add_subplot(311)
ax1.plot(x,x)

ax2 = fig.add_subplot(312,sharex=ax1)
ax2.plot(x*2,x*2)

ax3 = fig.add_subplot(313,sharex=ax1)
ax3.plot(x*3,x*3)

plt.show()

效果图

绘制日期

使用plot_date绘制坐标轴为日期的图形。
下面的linestyle会将各个随机点连接起来，下图有个缺点就是x的日期坐标轴会覆盖。

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np
import datetime as dt

dates = [dt.datetime.today() + dt.timedelta(days=i) for i in range(10)]

values = np.random.rand(len(dates))

plt.plot_date(mpl.dates.date2num(dates),values,linestyle='-')

plt.show()

效果图

日期格式化

date的格式如下所示：

1

datetime.datetime(2008, 7, 18, 14, 36, 53, 494013)

为了能让Matplotlib使用日期，我们需要将其更改为浮点类型。

可以参考下面的函数：

• date2num
• num2date
• drange

drange比较有用的情况：

1
2
3
4
5
6
7

import matplotlib as mpl
from matplotlib import dates
import datetime as dt
date1 = dt.datetime(2008, 9, 23)
date2 = dt.datetime(2009, 4, 12)
delta = dt.timedelta(days=10)
dates = mpl.dates.drange(date1, date2, delta)

前面绘制的图片比较丑，是真丑，x轴坐标完全看不到。

此时我们就需要使用locator和formatter来格式化坐标轴。其中：

• locator : control the tick’s position
• formatter : control the formatting of labels

因为matplotlib有默认的设置，所以最好将这些设置放在plot_date函数的后面，
以防被其重置为原来的值。

解释：

• fig.autofmt_xdate() 用于美化调整日期显示格式，会旋转以防文字重叠
• subplots_adjust() 用于调整plot的空间，可以使用bottom，top，left和right关键词以及wspace和hspace关键词

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np
import datetime as dt

fig = plt.figure()

ax2 = fig.add_subplot(212)
date2_1 = dt.datetime(2008,9,23)
date2_2 = dt.datetime(2008,10,3)
delta2 = dt.timedelta(days=1)
dates2 = mpl.dates.drange(date2_1,date2_2,delta2)
y2 = np.random.rand(len(dates2))
ax2.plot_date(dates2,y2,linestyle='-')
dateFmt = mpl.dates.DateFormatter('%Y-%m-%d')
ax2.xaxis.set_major_formatter(dateFmt)
daysLoc = mpl.dates.DayLocator()
hoursLoc = mpl.dates.HourLocator(interval = 6)
ax2.xaxis.set_major_locator(daysLoc)
ax2.xaxis.set_minor_locator(hoursLoc)
fig.autofmt_xdate()

fig.subplots_adjust(left=0.18)
ax1 = fig.add_subplot(211)
date1_1 = dt.datetime(2008,9,23)
date1_2 = dt.datetime(2009,2,16)
delta1 = dt.timedelta(days=10)
dates1 = mpl.dates.drange(date1_1,date1_2,delta1)
y1 = np.random.rand(len(dates1))
ax1.plot_date(dates1,y1,linestyle='-')
monthsLoc = mpl.dates.MonthLocator()
weeksLoc = mpl.dates.WeekdayLocator()
ax1.xaxis.set_major_locator(monthsLoc)
ax1.xaxis.set_minor_locator(weeksLoc)
monthsFmt = mpl.dates.DateFormatter('%b')
ax1.xaxis.set_major_formatter(monthsFmt)

plt.show()

效果图

Qt4中潜入Matplotlib

Qt应该是跨平台做的算是很好的了，一次编辑，多次编译跨平台。

这一章会介绍：

• 把Matplotlib的Figure对象嵌入到Qt的组件中
• 把Matplotlib的Figure和导航工具栏嵌入到Qt的组件中
• 使用时间来实时更新Matplotlib
• 使用Qt Designer来绘制GUI，然后调用程序使用Matplotlib

Qt4和PyQt4的简介

Qt是一个跨平台的开发框架，被广泛用于GUI的开发，最有名的应当是KDE桌面环境了。

其实Qt不仅仅是一个GUI开发工具包，它还包含一系列的网络、线程、Unicode、正则表达式、数据库、OpenGL和XML等库。

Qt是跨平台的，可以用与Unix/Linux，Windows，MacOSX。

尽管Qt是用C++来编写的，也可以通过绑定来使用其他语言，诸如Ruby，Java，Perl和Python。

PyQt绑定Qt2和Qt3，PyQt4绑定的是Qt4。

几个比较重要的Qt模块：

• QtCore：非GUI类
• QtGui：GLI类
• QtOpenGL，QtSql，QtSvg，QtText，QtXml，QtNetwork等

嵌入一个Matplotlib Figure到一个Qt窗口

几个需要注意的事情：

• from matplotlib.figure import Figure ：这里的Figure是我们绘图的后端独立单元
  from matplotlib.backends.backend_qt4agg import FigureCanvasQTAgg as FigureCanvas ：这里的FigureCanvas使我们绘图的画布，并且除了是一个Matplotlib类外，还是一个QWidget类，所以可以直接继承使用。

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import sys
from PyQt4 import QtGui
import numpy as np
from matplotlib.figure import Figure
from matplotlib.backends.backend_qt4agg import FigureCanvasQTAgg as FigureCanvas

class Qt4MplCanvas(FigureCanvas):
    '''
    Class to represent the FigureCanvas widget
    '''
    def __init__(self):
        self.fig = Figure()
        self.axes = self.fig.add_subplot(111)
        self.x = np.arange(0.,3.,0.01)
        self.y = np.cos(2*np.pi*self.x)
        self.axes.plot(self.x,self.y)

        # initialize the canvas where the Figure renders into
        FigureCanvas.__init__(self,self.fig)

qApp = QtGui.QApplication(sys.argv)
mpl = Qt4MplCanvas()
mpl.show()
sys.exit(qApp.exec_())

效果图

PyQt4中嵌入Figure和导航条

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

#!/usr/bin/env python
# coding=utf-8

# Author    :   Guo Shaoguang
# Email     :   sgguo@shao.ac.cn
# Institute :   Shanghai Astronomical Observatory

import sys
from PyQt4 import QtGui
import numpy as np
from matplotlib.figure import Figure
from matplotlib.backends.backend_qt4agg import FigureCanvasQTAgg as FigureCanvas
from matplotlib.backends.backend_qt4agg import NavigationToolbar2QT as NavigationToolbar

class Qt4MplCanvas(FigureCanvas):
    '''
    Class to represent the FigureCanvas widget
    '''
    def __init__(self,parent):
        self.fig = Figure()
        self.axes = self.fig.add_subplot(111)
        self.x = np.arange(0.,3.,0.01)
        self.y = np.cos(2*np.pi*self.x)
        self.axes.plot(self.x,self.y)

        # initialize the canvas where the Figure renders into
        FigureCanvas.__init__(self,self.fig)

        self.setParent(parent)

        FigureCanvas.setSizePolicy(self,QtGui.QSizePolicy.Expanding,QtGui.QSizePolicy.Expanding)
        FigureCanvas.updateGeometry(self)

class ApplicationWindow(QtGui.QMainWindow):
    def __init__(self):
        QtGui.QMainWindow.__init__(self)
        self.setWindowTitle("Matplotlib Figure in a Qt4 Window With NavigationToolbar")
        self.main_widget = QtGui.QWidget(self)
        vbl = QtGui.QVBoxLayout(self.main_widget)
        qmc = Qt4MplCanvas(self.main_widget)
        ntb = NavigationToolbar(qmc,self.main_widget)

        vbl.addWidget(qmc)
        vbl.addWidget(ntb)

        self.main_widget.setFocus()
        self.setCentralWidget(self.main_widget)


qApp = QtGui.QApplication(sys.argv)
aw = ApplicationWindow()
aw.show()
sys.exit(qApp.exec_())

效果图

直方图

源码

效果图

scatter简介

基本用法

1

scatter(x, y, s=None, c=None, marker=None, cmap=None, norm=None, vmin=None, vmax=None, alpha=None, linewidths=None, verts=None, edgecolors=None, hold=None, data=None, **kwargs)

其中各个参数的含义如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

x，y： 输入数据

s：指定散点的大小

c：指定散点的颜色。

marker：指定散点的图形样式。应参数支持'.'（点标记）、','（像素标记）、'o'（圆形标记）、'v'（向下三角形标记）、'^'（向上三角形标记）、'<'（向左三角形标记）、'>'（向右三角形标记）、'1'（向下三叉标记）、'2'（向上三叉标记）、'3'（向左三叉标记）、'4'（向右三叉标记）、's'（正方形标记）、'p'（五地形标记）、'*'（星形标记）、'h'（八边形标记）、'H'（另一种八边形标记）、'+'（加号标记）、'x'（x标记）、'D'（菱形标记）、'd'（尖菱形标记）、'|'（竖线标记）、'_'（横线标记）等值。

cmap：指定散点的颜色映射，会使用不同的颜色来区分散点的值

norm：`〜matplotlib.colors.Normalize`，可选，默认：无
       `〜matplotlib.colors.Normalize`实例用于缩放
       亮度数据为0,1。`norm`只有在`c`是一个数组时才被使用
       彩车。如果`None'，则使用默认值：func：`normalize`。

vmin，vmax：标量，可选，默认值：无
       `vmin`和`vmax`与`norm`结合使用来标准化
       亮度数据。如果其中任何一个都是`无'，那么最小和最大的
       使用颜色数组。请注意，如果你通过一个“规范”实例，你的
       `vmin`和`vmax`的设置将被忽略。

alpha：指定散点的透明度，介于0（透明）和1（不透明）之间，

linewidths：指定散点边框线的宽度

verts：（x，y）的序列，可选
       如果`marker`为None，这些顶点将用于
       构建标记。标记的中心位于
       在（0,0）为标准化单位。整体标记重新调整
       由``s``完成。

edgecolors ：指定散点边框的颜色

示例图1

从原点开始逐渐增加点的大小

1
2
3
4
5

x = [i for i in np.arange(0,10,1)]
y = [0]*len(x)
s = [3**n for n in range(len(x))]
plt.scatter(x,y,s=s)
plt.show()

示意图2

展示一个随机散点图

1
2
3
4
5
6
7
8
9
10
11
12

np.random.seed(5565)


N = 50
x = np.random.rand(N)
y = np.random.rand(N)
colors = np.random.rand(N)
area = (30 * np.random.rand(N))**2  # 0 to 15 point radii

plt.scatter(x, y, s=area, c=colors, alpha=0.5)

plt.show()

示意图3

画个同心圆

1
2
3
4
5
6

plt.figure(figsize=(8,6))
plt.scatter(2, 1, s=4000, c='r')
plt.scatter(2, 1, s=1000 ,c='b')
plt.scatter(2, 1, s=10, c='g')

plt.show()

示意图4

带标记

1
2
3
4
5
6
7
8
9
10
11

plt.figure(figsize=(8,6))
x_coords = [0.13, 0.22, 0.39, 0.59, 0.68, 0.74,0.93]
y_coords = [0.75, 0.34, 0.44, 0.52, 0.80, 0.25,0.55]

plt.scatter(x_coords, y_coords, marker = 's', s = 50)
for x, y in zip(x_coords, y_coords):
    plt.annotate('(%s,%s)'%(x,y), xy=(x,y),xytext = (0, -10), textcoords = 'offset points',ha = 'center', va = 'top')
plt.xlim([0,1])
plt.ylim([0,1])

plt.show()

其他示意图

标记的示意图

两个分类的示意图

https://www.in-ulm.de/~mascheck/various/argmax/

ARG_MAX, maximum length of arguments for a new process
or why do you get
command: arg list too long
2002-06-06 .. 2016-09-07 (see recent changes)

Here you’ll find

More about the nature of this limit
The effectively usable space …and a way to determine it reliably
…alternatively: about the GNU autoconf check
How to avoid the limit in a shell
other limits: number of arguments and maximum length of one argument
Actual values for ARG_MAX (with some more details in the footnotes)
More pages
More about the nature of this limit
You will see this error message, if you try to call a program with too many arguments, that is,
most likely in connection with pattern matching:

$ command *

On some systems the limit is even hit with “grep pattern /usr/include//“ (apart from that using find would be more appropriate).

It’s only the exec() system call and its direct variants, which will yield this error.
They return the corresponding error condition E2BIG (<sys/errno.h>).
The shell is not to blame, it just delivers this error to you.
In fact, shell expansion is not a problem, because here exec() is not needed, yet.
Expansion is only limited by the virtual memory system resources [1].

Thus the following commands work smoothly, because instead of handing over too many arguments to a new process,
they only make use of a shell built-in (echo) or iterate over the arguments with a control structure (for loop):

/dir-with-many-files$ echo * | wc -c
/dir-with-many-files$ for i in * ; do grep ARG_MAX "$i"; done

There are different ways to learn the upper limit

command: getconf ARG_MAX [2]
system call: sysconf(_SC_ARG_MAX) [3]
system header: ARG_MAX in e.g. <[sys/]limits.h> [4]
try xargs –show-limits[5], if you use GNU xargs
(However, on the few system that have no limit for ARG_MAX, these methods wrongly might print a limit.)

From Version 7 on the limit was defined by NCARGS (usually in <[sys/]params.h>),
Later, ARG_MAX was introduced with 4.4BSD and System V.

In contrast to the headers, sysconf and getconf tell the limit which is actually in effect.
This is relevant on systems which allow to change it at run time (AIX), by reconfiguration (UnixWare, IRIX),
by recompiling (e.g. Linux) or by applying patches (HP-UX 10) - see the end notes for more details.
(Usually these are solutions for special requirements only, because increasing the limit doesn’t solve the problem.)

[1]	However, in contrast to such expansions (which includes the literal overall command line length in scripts),

shells do have a limit for the interactive command line length (that is, what you may type in after the prompt).
But this limit is shell specific and not related to ARG_MAX.
Interestingly, putenv(3) is only limited by system resources, too. You just can’t exec() anmymore if you are over the limit.
[2] 4.4BSD BSD and the successors ( NetBSD since 1.0, OpenBSD 2.0, FreeBSD 2.0 ) provide: sysctl kern.argmax.
getconf in turn was introduced on BSDs with these versions: NetBSD 1.0, OpenBSD 2.0, FreeBSD 4.8.
[3] example usage of sysconf():
#include <stdio.h>
#include <unistd.h>
int main() {
return printf(“ARG_MAX: %ld\n”, sysconf(_SC_ARG_MAX));
}
[4] A handy way to find the limits in your headers, if you have cpp(1) installed, which doesn’t abort on file not found,
(inspired by Era Eriksson’s page about ARG_MAX):

cpp <<EOF
#include <limits.h>
#include <param.h>
#include <params.h>
#include <sys/limits.h>
#include <sys/param.h>
#include <sys/params.h>
arg_max: ARG_MAX
ncargs: NCARGS
EOF
If your cpp doesn’t like non-existent files, you might try

for file in limits.h param.h params.h sys/limits.h sys/param.h sys/params.h linux/limits.h linux/limits.h linux/param.h; do
cpp <<EOF 2>&1
#include <$file>
arg_max: ARG_MAX
ncargs: NCARGS
EOF
done|egrep ‘arg_max|ncargs’ |egrep -v ‘ARG_MAX|NCARGS’

[5]	$ xargs --show-limits

environment variables take up 533 bytes
POSIX upper limit on argument length (this system): 2094571
POSIX smallest allowable upper limit on argument length (all systems): 4096
Maximum length of command we could actually use: 2094038
Size of command buffer we are actually using: 131072
The effectively usable space
When looking at ARG_MAX/NCARGS, you have to consider the space comsumption by both argv[] and envp[] (arguments and environment).
Thus you have to decrease ARG_MAX at least by the results of “env|wc -c” and “env|wc -l * 4” [5] for a good estimation of the currently available space.

[5]	 Every entry in envp is terminated with a null byte. The env utility adds a terminating newline instead, so the result of "wc -c" is the same.

“wc -l” in turn accounts for the number of pointers in envp, i.e., usually 4 bytes each, according to sizeof().
Some modern shells allow for exporting functions to the environment. The above slightly miscalculates then,
because their definitions tend to contain newlines which are misinterpreted as new envp[].
The same applies if variable values contain newlines.
You can make wc -l ignore the wrappings and limit it to lines with = at the right place:
expr getconf ARG_MAX - env|wc -c - env|egrep '^[^ ]+='|wc -l * 4
(thanks to Michael Klement for pointing out the function issue and improving the calculation)

POSIX suggests to subtract 2048 additionally so that the process may savely modify its environment. A quick estimation with the getconf command:
(all the calculations inspired by a post from Gunnar Ritter in de.comp.os.unix.shell, 3B70A6AD.3L8115910@bigfoot.de)

expr getconf ARG_MAX - env|wc -c - env|wc -l * 4 - 2048
or, if you even want to consider wrapped functions or variable values [5],
expr getconf ARG_MAX - env|wc -c - env|egrep '^[^ ]+='|wc -l * 4 - 2048,
…and a way to determine it reliably
The most reliable way to get the currently available space is to test the success of an exec() with increasing length of arguments until it fails.
This might be expensive, but at least you need to check only once, the length of envp[] is considered automatically, and the result is reliable.
…alternatively: about the GNU autoconf check
There’s an autoconf check “Checking for maximum length of command line arguments…”. It works quite similar.
However, it results in a much lower value (it can be a fourth of the actual value only) both by intention and for reasons of simplicity:
In a loop with increasing n, the check tries an exec() with an argument length of 2n (but won’t check for n higher than 16, that is 512kB).
The maximum is ARG_MAX/2 if ARG_MAX is a power of 2.
Finally, the found value is divided by 2 (for safety), with the reason “C++ compilers can tack on massive amounts of additional arguments”.

How to avoid the limit in a shell
If command * fails, then you can
iterate with the shell:
for i in *; do command “$i”; done (simple, completely robust and portable, may be very slow)
printf ‘%s\0’ *|xargs -0 command (works only if printf is a built-in, but then it can be much faster on high counts. thanks to Michael Klement)
iterate with find
find . -exec command {} ; (simple, completely robust and portable, may be very slow)
find . -exec command {} + (optimizes speed, quite portable)
find . -print0|xargs -0 command (optimizes speed, if find doesn’t implement “-exec +” but knows “-print0”)
find . -print|xargs command (if there’s no white space in the arguments)

Note: find descends into directories. To avoid that portably, you can use
“find . ! -name . -prune […]”
If the major part of the arguments consists of long, absolute or relative paths, then try to move your actions into the directory:
cd /directory/with/long/path; command *
And another quick fix may be to match fewer arguments:
command [a-e]; command [f-m]; …
Number of arguments and maximum length of one argument
At least on Linux 2.6, there’s also a limit on the maximum number of arguments in argv[].
On Linux 2.6.14 the function do_execve() in fs/exec.c tests if the number exceeds
PAGE_SIZE*MAX_ARG_PAGES-sizeof(void *) / sizeof(void *)
On a 32-bit Linux, this is ARGMAX/4-1 (32767). This becomes relevant if the average length of arguments is smaller than 4.
Since Linux 2.6.23, this function tests if the number exceeds MAX_ARG_STRINGS in <linux/binfmts.h> (2^32-1 = 4294967296-1).

And as additional limit since 2.6.23, one argument must not be longer than MAX_ARG_STRLEN (131072).
This might become relevant if you generate a long call like “sh -c ‘automatically generated with many arguments’”.
(pointed out by Xan Lopez and Ralf Wildenhues)

Actual values for ARG_MAX (or NCARGS)
The maximum length of arguments for a new process is varying so much among unix flavours, that I had a look at some systems:

System value getconf
available default value determined by
non-competitive: 1st edition (V1) 255+? [1stEd] experiments
non-competitive: V4, V5 and V6 512 documentation of exec(2) in V4, V6 and (no manual) sys1.c in V5
Version 7,
3 BSD,
System III, SVR1,
Ultrix 3.1 5120 NCARGS in <sys/param.h>
4.0/4.1/4.2 BSD 10240 NCARGS in <sys/param.h>
4.3 BSD / and -Tahoe 20480 NCARGS in <sys/syslimits.h>
4.3BSD-Reno, 4.3BS-Net2
4.4 BSD (alpha/lite/encumbered),
386BSD*, NetBSD 0.9,
BSD/OS 2.0 20480 ARG_MAX in <sys/syslimits.h> (NCARGS in <sys/param.h>)
POSIX/SUSv2,v3,v4 [posix] 4096 (minimum) + minimum _POSIX_ARG_MAX in <limits.h> , ARG_MAX
AIX 3.x, 4.x, 5.1[aix5] 24576 + ARG_MAX in <sys/limits.h> (NCARGS in <sys/param.h>)
AIX 6.1, 7.2 1048576 + online documentation (FilesReference/HeaderFiles) 6.1, 7.2 (ARG_MAX in <limits.h>)
BSD/OS 4.1,
NetBSD 1.0+x,
OpenBSD x: 262144 + ARG_MAX(/NCARGS) in <sys/syslimits.h>
Cygwin 1.7.7 (win 5.1) [cygwin] 30000 ARG_MAX in <limits.h>
Dynix 3.2 12288 ARG_MAX in <(sys/)limits.h> (NCARGS in <sys/param.h>)
EP/IX 2.2.1AA: 20480 ARG_MAX in <sys/limits.h>
FreeBSD 2.0-5.5 65536 + ARG_MAX(/NCARGS) in <sys/syslimits.h> [freebsd]
FreeBSD 6.0 (PowerPC 6.2, ARM 6.3) 262144 + ARG_MAX(/NCARGS) in <sys/syslimits.h> [freebsd]
GNU Hurd 0.3 Mach 1.3.99 unlimited [hurd]
(stack size?) +
Haiku OS (2008-05-14) [haiku] 131072 ? MAX_PROCESS_ARGS_SIZE in <system/user_runtime.h>
HP-UX 8(.07), 9, 10 20478 + ARG_MAX in <limits.h>
HP-UX 11.00 2048000 [hpux] + ARG_MAX in <limits.h>
Interix 3.5 1048576 + -
IRIX 4.0.5 10240 NCARGS in <sys/param.h> (fallback: ARG_MAX in <limits.h>: 5120)
IRIX 5.x, 6.x 20480 [irix] + (fallback: ARG_MAX in <limits.h>: 5120)
Linux -2.6.22 131072 + ARG_MAX in <linux/limits.h> [linux-pre-2.6.23]
Linux 2.6.23 (1/4th of stack size) + kernel code [linux-2.6.23]
MacOS X 10.6.2 (xnu 1486.2.11) 262144 + ARG_MAX(/NCARGS) in <sys/syslimits.h>
MUNIX 3.2 10240 ? ARG_MAX in <sys/syslimits.h>
Minix 3.1.1 16384 ARG_MAX in <limits.h>
OSF1/V4, V5 38912 + ARG_MAX in <sys/syslimits.h>
SCO UNIX SysV R3.2 V4.0/4.2
SCO Open Desktop R2.0/3.0 5120 ? online documentation
SCO OpenServer 5.0.x [osr5] 1048576 + (fallback: ARG_MAX in <limits.h>: 5120)
UnixWare 7.1.4,
OpenUnix 8 32768 [uw/osr6] + (fallback ARG_MAX in <limits.h>: 10240)
SCO OpenServer 6.0.0 32768 [uw/osr6] + (fallback: ARG_MAX in <limits.h>: 10240)
SINIX V5.2 10240 ? ARG_MAX in <limits.h>
SunOS 3.x 10240 ? ARG_MAX in <sys/param.h>
SunOS 4.1.4 1048576 NCARGS in <sys/param.h> , sysconf(_SC_ARG_MAX)
SunOS 5.x (32bit process) 1048320 [sunos5] + ARG_MAX in <limits.h> (NCARGS in <sys/param.h>)
SunOS 5.7+ (64bit process) 2096640 [sunos5] + ARG_MAX in <limits.h> (NCARGS in <sys/param.h>)
SVR4.0 v2.1 (386) 5120 ? (no ARG_MAX/NCARGS in in <limits.h>/<sys/param.h>)
Ultrix 4.3 (vax / mips) 10240 / 20480 NCARGS in <sys/param.h>
Unicos 9,
Unicos/mk 2 49999 + ARG_MAX in <sys/param.h>
UnixWare 7: see OpenServer 6
UWIN 4.3 AT&T Unix Services for Windows 32768 + ARG_MAX in <limits.h>
[posix] See the online documentation (please register for access) for getconf and <limits.h>.
[osr5] Bela Lubkin points out:
The limit on SCO OpenServer 5.0.x is set by ‘unsigned int maxexecargs = 1024*1024;’
in /etc/conf/pack.d/kernel/space.c. It can also be changed on a live system with the scodb kernel debugger:

scodb -w

scodb> maxexecargs=1000000
scodb> q
(0x1000000 = 16MiB.) This is the max size of a new temporary allocation during each exec(), so it’s safe to change on the fly.

Exceeding the limit generates a kernel warning:

WARNING: table_grow - exec data table page limit of 256 pages (MAXEXECARGS) exceeded by 1 pages
WARNING: Reached MAXEXECARGS limit while adding arguments for executable “ls”
Some configure scripts trigger this message as they deliberately probe the limit.
Raising `maxexecargs’ will not fix this as the probe will simply try harder.

[uw/osr6] The limit on UnixWare can be increased by changing the kernel parameter ARG_MAX with /etc/conf/bin/idtune,
(probably in the range up to 1MB) regenerating the kernel with “etc/conf/bin/idbuild -B” and rebooting.
See also the online documentation.
On UnixWare 7.1.4, the run time limit for a default install of “Business Edition” is 32768.

Bela Lubkin points out, that, very basically, OpenServer 6 can be described as a UnixWare 714 kernel with the OpenServer 5.0.7 userland running on top of it.

[irix] The limit on IRIX can be changed by changing the kernel parameter ncargs with systune
(in the range defined in /var/sysgen/mtune/kernel, probably varying from 64KB to 256KB),
regenerating the kernel with “autoconfig” and rebooting. See also the online documentation of systune(1M) and intro(2).
[aix5] The limit on AIX 5.1 can be changed at run time with “chdev -l sys0 -a ncargs=value”, in the range from 64KB to 10244KB.
See also the online documentation for chdev (AIX documentation, Commands reference).
[freebsd] Interesting and everything but academic was the reason for the first of two increases (40960, 65536) on FreeBSD:
“Increase ARG_MAX so that `make clean’ in src/lib/libc works again.
(Adding YP pushed it over the limit.)”
quoted from http://www.FreeBSD.org/cgi/cvsweb.cgi/src/sys/sys/syslimits.h
[linux-pre-2.6.23] On Linux, the maximum almost always has been PAGE_SIZEMAX_ARG_PAGES (409632) minus 4.
However, in Linux-0.0.1, ARG_MAX was not known yet, E2BIG not used yet and exec() returned -1 instead.
With linux-0.10 it returned ENOMEM and with Linux-0.99.8 it returned E2BIG.
ARG_MAX was introduced with linux-0.96, but it’s not used in the kernel code itself.
See do_execve() in fs/exec.c on http://www.oldlinux.org/Linux.old/.
If you want to increase the limit, you might succeed by carefully increasing MAX_ARG_PAGES (link to a discussion on the linux kernel mailing list 03/‘00)

[linux-2.6.23] With Linux 2.6.23, ARG_MAX is not hardcoded anymore. See the git entry.
It is limited to a 1/4-th of the stack size (ulimit -s), which ensures that the program still can run at all.
See also the git diff of fs/exec.c
getconf ARG_MAX might still report the former limit (being careful about applications or glibc not catching up, but especially because the kernel <limits.h> still defines it)

[sunos5] On SunOS 5.5, according to <limits.h>, ARG_MAX is 1M, decreased by the following amount:
“((sizeof(struct arg_hunk ))(0x10000/(sizeof)(struct arg_hunk)))
space for other stuff on initial stack like aux vectors, saved
registers, etc..”
On SunOS 5.9 this reads
“ARG_MAX is calculated as follows:
NCARGS - space for other stuff on initial stack
like aux vectors, saved registers, etc..”
and <sys/param.h> defines NCARGS32/64 to 0x100000/0x200000 with NCARGS being substited at compile time.
ARG_MAX is not calculated in the header files but is set directly in <limits.h>, also substitued at compile time from _ARG_MAX32/64.
SunOS 5.7 is the first release to support 64bit processes.

[hpux] HP-UX 11 can also run programs compiled on HP-UX 10. Programs which have ARG_MAX compiled in as buffer length
and copy from argv[]/envp[] without boundary checking might crash due to the increased ARG_MAX.
See devresource.hp.com
[hurd] NCARGS in contrast is arbitrarily set to INT_MAX (2147483647) in <i386-gnu/sys/param.h>
The reason: “ARG_MAX is unlimited, but we define NCARGS for BSD programs that want to compare against some fixed limit.”
I don’t know yet, if there are other limits like the stack.
[cygwin] ARG_MAX 32000 was added to <limits.h> on 2006-11-07. It’s a conservative value, having in mind the windows limit of 32k.
However, the cygwin internal limit, that is, if you don’t call non-cygwin binaries, is much higher.
[haiku] “Haiku is an open-source operating system […] inspired by the BeOS” (www.haiku-os.org). Thanks to Sylvain Kerjean for this pointer!
Note that there is also <posix/limits.h> with ARG_MAX / _POSIX_ARG_MAX for sysconf(), with more a more conservative value of 32768.
[1stEd] By judging from experiments in the simh emulator with 1st edition kernel and 2nd edition shell, the results are somewhat undefined.
If the length or number of arguments (there is no environment yet) is too high, data corruption may occur, including a kernel crash.
The following may or may not indicate the nature of limits:
From the BUGS section in the 3rd edition exec(2) manual:
Very high core and very low core are used by exec to construct the argument list for the new core image.
If the original copies of the arguments reside in these places, problems can result.
and a related information about the placement of the arguments
(which is also available in 1st ed manual) reads equivalent:
1st edition: The arguments are placed as high as possible incore: just below 60000(8).
3rd edition: The arguments are placed as high as possible in core: just below 57000(8).

By calling a script which just echoes its arguments (“sh s arguments”), I found:

• command line (script or interactive) not longer than 255 characters
• single argument not longer than 82 characters
  As there is no working compiler (B) on that system, I haven’t digged further.

Visual Studio Code

Extensions

• C/C++
• Chinese ( Simplified) Language Pack for Visual Studio Code
• Doxygen Documentation Generator
• Markdown All in One
• Python
• TODO Highlight
• Vim
• Wakatime
• YAML Support by Red Hat
• vscode-icons
• autoconf
• LaTex Workshop

添加 Visual Studio Code 命令行到 Mac OS X 环境变量

在环境变量中加入下面一行，或者bashrc亦或者zsh，然后就可以在终端输入code来启动了。

1

code () { VSCODE_CWD="$PWD" open -n -b "com.microsoft.VSCode" --args $* ;}

增加xelatex用于编译中文文章

只需要在tex文件的最前面，加上% !TEX program = xelatex，在编译的时候就会直接调用xelatex。

1
2
3
4
5
6
7
8
9
10
11
12

% !TEX program = xelatex
\documentclass{article}

\author{Hello}
\title{Test of xelatex in VSCode}

\begin{document}
\maketitle

你好，世界！

\end{document}