王鸣辉的博客

Wang Minghui's Personal Blog

caffe_digits

• 标签: OS X 10.12, caffe, digits

最近定了下方向，要搞视频理解方面的东西。因为兄弟部门用caffe做了很多东西，于是打算从caffe入手先去学习学习

在mac上部署caffe + digits

补充

迫于无奈，手里只有一个mbp，于是只能在mac上装了，mac上装比较麻烦，尤其是OS X 10.12。建议如果有条件的话还是使用ubuntu吧，能省事很多很多

caffe 安装

参考 Eddie Smolyansky’s Blog

参考

之前按官网推荐安装了anaconda，但官网文档已经比较旧了，据说用这个阻碍其实会更多，果断卸载之。

换用virtualenvwrapper,很轻便

然后安装cuda cudnn(我装的8.0.51版本，后边也会有地方和版本对应)

cudnn需要注意下，解压后是cuda目录
把对应的lib64、include目录里的东西copy到/usr/local/cuda里

1
2
cp include/* /usr/local/cuda/include/*
cp lib64/* /usr/local/cuda/lib64/*

然后执行install_caffe.sh

按如下脚本执行，有错误就解决，补充以及常见问题方案

  1. 下载XCode Command Line Tools for 7.3, 因为NVIDIA 暂时还不支持Xcode 8.0的. 切换到7.3的tools:

    sudo xcode-select —switch /Library/Developer/CommandLineTools

待会还要删掉它并使用以下语句切回来, 不然你会用不了brew

sudo xcode-select -s /Applications/Xcode.app/Contents/Developer

  1. BLAS那边你可能会遇到问题，在Makefile.config里加如下几行
1
2
3
BLAS_INCLUDE := $(shell brew --prefix openblas)/include
BLAS_LIB := $(shell brew --prefix openblas)/lib
LIB_CUDA := -L/usr/local/cuda/lib -lcuda

  1. libcnn.so找不到，这个说起来我有，不知道为啥找不到，所以直接把
    USE_CUDNN := 1 注掉了

  2. 下边脚本执行可能还会有些问题，不过大多是缺东西，直接装就行了，然后对应的版本要调整，比如脚本里边是libcaffe.so.1.0.0-rc3，而我这边实际是libcaffe.so.1.0.0等

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96

#!/bin/sh
# Install brew
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
# Apple hides old versions of stuff at https://developer.apple.com/download/more/
# Install the latest XCode (8.0).
#   We used to install the XCode Command Line Tools 7.3 here, but that would just upset the most recent versions of brew.
#   So we're going to install all our brew dependencies first, and then downgrade the tools. You can switch back after
#   you have installed caffe.
# Install CUDA toolkit 8.0 release candidate
#   Register and download from https://developer.nvidia.com/cuda-release-candidate-download
#   or this path from https://developer.nvidia.com/compute/cuda/8.0/rc/local_installers/cuda_8.0.29_mac-dmg
#   Select both the driver and the toolkit, no documentation necessary
# Install the experimental NVIDIA Mac drivers
#   Download from http://www.nvidia.com/download/driverResults.aspx/103826/en-us
# Install cuDNN v5 for 8.0 RC or use the latest when it's available
#   Register and download from https://developer.nvidia.com/rdp/cudnn-download 
#   or this path: https://developer.nvidia.com/rdp/assets/cudnn-8.0-osx-x64-v5.0-ga-tgz
#   extract to the NVIDIA CUDA folder and perform necessary linking
#   into your /usr/local/cuda/lib and /usr/local/cuda/include folders
#   You will need to use sudo because the CUDA folder is owned by root
sudo tar -xvf ~/Downloads/cudnn-8.0-osx-x64-v5.0-ga.tar /Developer/NVIDIA/CUDA-8.0/
sudo ln -s /Developer/NVIDIA/CUDA-8.0/lib/libcudnn.dylib /usr/local/cuda/lib/libcudnn.dylib
sudo ln -s /Developer/NVIDIA/CUDA-8.0/lib/libcudnn.5.dylib /usr/local/cuda/lib/libcudnn.5.dylib
sudo ln -s /Developer/NVIDIA/CUDA-8.0/lib/libcudnn_static.a /usr/local/cuda/lib/libcudnn_static.a
sudo ln -s /Developer/NVIDIA/CUDA-8.0/include/cudnn.h /usr/local/cuda/include/cudnn.h
# Install the brew dependencies
#   Do not install python through brew. Only misery lies there
#   We'll use the versions repository to get the right version of boost and boost-python
#   We'll also explicitly upgrade libpng because it's out of date
#   Do not install numpy via brew. Your system python already has it.

brew install -vd snappy leveldb gflags glog szip lmdb
brew install hdf5 opencv
brew upgrade libpng
brew tap homebrew/versions

brew install --build-from-source --with-python -vd protobuf
brew install --build-from-source -vd boost159 boost-python159

# Clone the caffe repo
cd ~/Documents
git clone https://github.com/BVLC/caffe.git
# Setup Makefile.config
#   You can download mine directly from here, but I'll explain all the selections
#     For XCode 7.3:
#       https://www.dropbox.com/s/vuy6ha0p7cc5px3/Makefile.config?dl=1
#     For XCode 8.0 and later (Sierra):
#       https://dl.dropboxusercontent.com/u/2891540/caffe_10.12/Makefile.config
#   First, we'll enable cuDNN
#     USE_CUDNN := 1
#   In order to use the built-in Accelerate.framework, you have to reference it.
#   Astonishingly, nobody has written this anywhere on the internet.
#     BLAS := atlas
#     If you use El Capitan (10.11), we'll use the 10.11 sdk path for vecLib:
#       BLAS_INCLUDE := /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/Headers
#     Otherwise (10.12), let's use the 10.12 sdk path:
#       BLAS_INCLUDE := /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/Headers
#     BLAS_LIB := /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A
#   Configure to use system python and system numpy
#     PYTHON_INCLUDE := /System/Library/Frameworks/Python.framework/Headers \
#          /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include
#     PYTHON_LIB := /System/Library/Frameworks/Python.framework/Versions/2.7/lib
#   Configure to enable Python layers. Some projects online need this
#     WITH_PYTHON_LAYER := 1
curl https://dl.dropboxusercontent.com/u/2891540/Makefile.config -o Makefile.config
# Download the XCode Command Line Tools for 7.3, since NVIDIA does not yet support Xcode 8.0's tools
#   http://adcdownload.apple.com/Developer_Tools/Command_Line_Tools_OS_X_10.11_for_Xcode_7.3/Command_Line_Tools_OS_X_10.11_for_Xcode_7.3.dmg
# Now, choose those tools instead
sudo xcode-select --switch /Library/Developer/CommandLineTools
# Go ahead and build.
make -j8 all
# To get python going, first we need the dependencies
#   On a super-clean Mac install, you'll need to easy_install pip.
sudo -H easy_install pip
#   Now, we'll install the requirements system-wide. You may also muck about with a virtualenv.
#   Astonishingly, --user is not better known. 
pip install --user -r python/requirements.txt
#   Go ahead and run pytest now. Horrible @rpath warnings which can be ignored.
make -j8 pytest
# Now, install the package
#   Make the distribution folder
make distribute
#   Install the caffe package into your local site-packages
cp -r distribute/python/caffe ~/Library/Python/2.7/lib/python/site-packages/
#   Finally, we have to update references to where the libcaffe libraries are located.
#   You can see how the paths to libraries are referenced relatively
#     otool -L ~/Library/Python/2.7/lib/python/site-packages/caffe/_caffe.so
#   Generally, on a System Integrity Protection -enabled (SIP-enabled) Mac this is no good.
#   So we're just going to change the paths to be direct
cp distribute/lib/libcaffe.so.1.0.0-rc3 ~/Library/Python/2.7/lib/python/site-packages/caffe/libcaffe.so.1.0.0-rc3
install_name_tool -change @rpath/libcaffe.so.1.0.0-rc3 ~/Library/Python/2.7/lib/python/site-packages/caffe/libcaffe.so.1.0.0-rc3 ~/Library/Python/2.7/lib/python/site-packages/caffe/_caffe.so
# Verify that everything works
#   start python and try to import caffe
python -c 'import caffe'
# If you got this far without errors, congratulations, you installed Caffe on a modern Mac OS X