Difference between revisions of "BuildTheCode"

From Sirikata Wiki
Jump to navigation Jump to search
(→‎Without Cygwin (manual): single submodule update command)
 
(15 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
= Dependencies =
 
= Dependencies =
  
Sirikata depends on quite a bit of external libraries. [[Dependencies]] gives a list of required and optional libraries and a brief description of how they are used.
+
Sirikata depends on quite a bit of external libraries. We provide install scripts and precompiled binaries to get you up and running quickly.
  
When building Sirikata you have two options: handle installing dependencies yourself or use our install scripts or precompiled binaries (depending on platform and library) to help you build and install the required libraries.  If you want to perform the installation or need to write new scripts for a new platform, see [[Dependencies]] for instructions on how to do so.
+
A makefile in the top-level directory provides commands for checking out the dependencies repository and installing them. While the <tt>depends</tt> target is usually what you want, there are a few variants you can use instead:
  
These instructions explain how to use our install script or precompiled binaries.
+
* <tt>minimal-depends</tt> - the minimal dependencies required to get the system building
 +
* <tt>minimal-graphics-depends</tt> - the minimal dependencies required to get 3D graphics working, i.e. to run a client, but not with all the bells and whistles
 +
* <tt>headless-depends</tt> - all dependencies that can be run headless, i.e. everything but the graphics support. Useful to avoid building expensive dependencies and plugins when you're going to be running on a server 
 +
* <tt>full-depends</tt> - all dependencies, enabling all features of the system, and enables use of root to install system packages
 +
 
 +
If you want more details, [[Dependencies]] gives a list of required and optional libraries and a brief description of how they are used.
  
 
{{note}} Some dependency files are quite large, especially the precompiled binaries.  When you run the install commands, it may appear the script has hung, but its likely just checking out the packages.
 
{{note}} Some dependency files are quite large, especially the precompiled binaries.  When you run the install commands, it may appear the script has hung, but its likely just checking out the packages.
 
{{note}} Throughout we'll be using the standard set of dependencies, which includes support for graphics, embedded browsers, scripting languages, and physics simulation via Bullet.  If you don't need all these features, for example because you are only running the space server, you can use a different dependency target.  Where <tt>depends</tt> is used, you can replace the following to get a different set of dependencies:
 
* <tt>minimal-depends</tt> - the minimal dependencies required to get the system building
 
* <tt>minimal-graphics-depends</tt> - the minimal dependencies required to get 3D graphics working, i.e. to get a client running.  Note that not all features of the client will be enabled by this
 
* <tt>full-depends</tt> - all dependencies, enabling all features of the system, and enables use of root to install system packages
 
  
 
== CMake ==
 
== CMake ==
Line 25: Line 25:
  
 
== Simple Method: Makefile ==
 
== Simple Method: Makefile ==
On all platforms, if you have the right tools (make and subversion, on Windows provided by Cygwin) a Makefile is provided in the root directory which checks out and either builds or extracts dependencies to <tt>sirikata/dependencies/</tt>:
+
On all platforms, if you have the right tools (make, subversion, and unzip on Windows provided by Cygwin) a Makefile is provided in the root directory which checks out and either builds or extracts dependencies to <tt>sirikata/dependencies/</tt>:
  
 
   cd sirikata/
 
   cd sirikata/
Line 33: Line 33:
 
The simple method requires Cygwin. All dependencies are installed locally in the Sirikata checkout.  No root access is needed.
 
The simple method requires Cygwin. All dependencies are installed locally in the Sirikata checkout.  No root access is needed.
  
{{note}} The Windows dependencies support Visual Studio 2008 and later, including Express editions. Visual Studio 2005 and earlier are no longer supported.
+
{{note}} The Windows dependencies support Visual Studio 2008 SP1 and later, including Express editions. You '''must''' have SP1 installed for Visual Studio 2008. Visual Studio 2005 and earlier are no longer supported.
  
 
=== Without Cygwin (manual) ===
 
=== Without Cygwin (manual) ===
Line 41: Line 41:
  
 
Go into the dependencies folder and unzip all of the packages directly into the dependencies directory. Make sure to select "Extract Here". If that option is not available, remove the name of the zip file from the end to indicate where to extract, so it just ends with "dependencies".  The extracted directories should not have the same name as the zip files.
 
Go into the dependencies folder and unzip all of the packages directly into the dependencies directory. Make sure to select "Extract Here". If that option is not available, remove the name of the zip file from the end to indicate where to extract, so it just ends with "dependencies".  The extracted directories should not have the same name as the zip files.
 +
 +
Then got to the top-level directory and run this command:
 +
git submodule update --init --recursive
  
 
== Mac ==
 
== Mac ==
  
=== Snow Leopard ===
+
=== Required Tools ===
  
With the release of Apple's Snow Leopard (10.6) operating system comes the opportunity to have software compiled for two different architectures running on the same machine. These are the 32-bit i386 and the 64-bit x86_64 (k8). Though the Intel hardware has always had the capability to run in a 64 bit address space prior to 10.6, the operating system didn't support it. Unfortunately, the default architecture is 64, but not all libraries and dependencies are 64-bit savvy, so it is best to compile for the 32 bit architecture.
+
You'll need a few required tools, most of which are built in. At a minimum, you'll need git and wget. These are easy to get from [http://mxcl.github.com/homebrew/ homebrew].
 +
 
 +
=== 32 vs. 64 bit ===
 +
 
 +
While other platforms (e.g. Linux) have supported 64 bit builds for awhile, currently many dependencies still don't support 64 bit builds on OS X. Because of this we force all builds into 32-bit i386 mode.
  
 
===== XCode and Developer Tools =====
 
===== XCode and Developer Tools =====
  
You need '''XCode Tools 3.2.x''' to build Sirikata. You can install these in parallel to XCode 4.0 if you already have that version. In particular, you need gcc-4.0 and g++-4.0. Version 4.2 won't work.
+
The transitions between Leopard, Snow Leopard, and Lion, gcc 4.0 and 4.2, and XCode 3 and 4 have made it quite challenging to support all configurations of OS X development machines. Here are the configurations which are known to work and are the suggested way to configure your environment:
 +
 
 +
* If you are on Lion, you should just install XCode 4. Currently it will include gcc 4.2.
 +
* If you are on Leopard or Snow Leopard you should use XCode 3.2.6 and gcc 4.0.
 +
 
 +
In both cases, when you run the dependency install script it should list whether it has found those configurations.
  
You can either download XCode at [http://developer.apple.com/mac] or use the Snow Leopard DVD: under “Optional Installs”, install “Xcode.mpkg” to get 3.2. Use all default options. We highly recommend at least 3.2.2 as previous versions had issues with hanging.
+
You can install XCode 3.2.x in parallel to XCode 4.0 if you already have that version.  You can either download XCode at [http://developer.apple.com/mac] or use the Snow Leopard DVD: under “Optional Installs”, install “Xcode.mpkg” to get 3.2. Use all default options. We highly recommend at least 3.2.2 as previous versions had issues with hanging.
  
 
{{note}} If you already installed XCode 4, then gcc and g++ were upgraded to version 4.2. The simplest course of action is to (re)install XCode 3.2 in parallel from [http://connect.apple.com/cgi-bin/WebObjects/MemberSite.woa/wo/5.1.17.2.1.3.3.1.0.1.1.0.3.3.3.3.1 here] -- the setup should work fine next to XCode 4.  You might be able to get by just by copying the old files (gcc-4.0 and g++-4.0) from /Developer-old/usr/bin back to /Developer/usr/bin, but this probably won't work for Snow Leopard (OS X 10.6).
 
{{note}} If you already installed XCode 4, then gcc and g++ were upgraded to version 4.2. The simplest course of action is to (re)install XCode 3.2 in parallel from [http://connect.apple.com/cgi-bin/WebObjects/MemberSite.woa/wo/5.1.17.2.1.3.3.1.0.1.1.0.3.3.3.3.1 here] -- the setup should work fine next to XCode 4.  You might be able to get by just by copying the old files (gcc-4.0 and g++-4.0) from /Developer-old/usr/bin back to /Developer/usr/bin, but this probably won't work for Snow Leopard (OS X 10.6).
Line 67: Line 79:
 
  libzzip-dev libxt-dev libxaw7-dev libxxf86vm-dev libxrandr-dev libfreetype6-dev \
 
  libzzip-dev libxt-dev libxaw7-dev libxxf86vm-dev libxrandr-dev libfreetype6-dev \
 
  libxext-dev autoconf libtool libpcre3-dev flex bison patch libbz2-dev gawk \
 
  libxext-dev autoconf libtool libpcre3-dev flex bison patch libbz2-dev gawk \
  libglu1-mesa-dev tofrodos libglut3-dev freeglut3-dev scons libexpat1-dev \
+
  libglu1-mesa-dev tofrodos freeglut3-dev scons libexpat1-dev \
 
  libgtk2.0-dev libnss3-dev libgconf2-dev gperf libasound2-dev subversion \
 
  libgtk2.0-dev libnss3-dev libgconf2-dev gperf libasound2-dev subversion \
 
  libtool autoconf ruby flex libgsl0-dev libssl-dev libspeex-dev libxss-dev \
 
  libtool autoconf ruby flex libgsl0-dev libssl-dev libspeex-dev libxss-dev \
 
  libdbus-glib-1-dev libgnome-keyring-dev libxml2-dev libjpeg62-dev libcups2-dev \
 
  libdbus-glib-1-dev libgnome-keyring-dev libxml2-dev libjpeg62-dev libcups2-dev \
  wget libcurl4-gnutls-dev
+
  wget libcurl4-gnutls-dev build-essential libxmu-dev libxi-dev libusb-1.0-0-dev \
 +
yasm
 +
 
 +
For Ubuntu 11.04 and earlier, add:
 +
 
 +
sudo apt-get install libglut3-dev
  
 
These packages should cover everything you need for the entire system, including the graphical client and all scripting plugins.
 
These packages should cover everything you need for the entire system, including the graphical client and all scripting plugins.
Line 86: Line 103:
 
On Windows Vista or later you might want to change CMAKE_INSTALL_PREFIX to the directory where you want to install your compiled binaries and hit configure again, because by default CMake will generate Visual Studio project that will try to install compiled binaries into %PROGRAMFILES%. This requires administrative privileges, which is why build will fail (unless you run Visual Studio with administrative privileges or have disabled UAC).
 
On Windows Vista or later you might want to change CMAKE_INSTALL_PREFIX to the directory where you want to install your compiled binaries and hit configure again, because by default CMake will generate Visual Studio project that will try to install compiled binaries into %PROGRAMFILES%. This requires administrative privileges, which is why build will fail (unless you run Visual Studio with administrative privileges or have disabled UAC).
  
Now browse to sirikata/build/cmake.  Open Sirikata.sln and run Build All.
+
Now browse to sirikata/build/cmake.  Open Sirikata.sln.  Select the configuration you want to build -- we highly recommend RelWithDebInfo because Debug runs very slowly and RelWithDebInfo still gets you useful information when run in the debugger. Finally, run Build All.
  
 
This should result in all libraries, plugins, and binaries in the sirikata/build/cmake/debug or sirikata/build/cmake release, depending on which configuration you built.
 
This should result in all libraries, plugins, and binaries in the sirikata/build/cmake/debug or sirikata/build/cmake release, depending on which configuration you built.
Line 94: Line 111:
 
* The build is known to work for VS2008, for both regular and Express versions.
 
* The build is known to work for VS2008, for both regular and Express versions.
 
* If you get error 0xc0000022, check the permissions of the sirikata top-level directory. If you used Cygwin's version of git, it will remove execute permissions for security purposes. You may enable execute by doing "chmod -R +x sirikata" in cygwin, or Granting yourself Full Control in Right Click->Properties->Security (make sure to click the Replace All Entries in Child Objects checkbox).
 
* If you get error 0xc0000022, check the permissions of the sirikata top-level directory. If you used Cygwin's version of git, it will remove execute permissions for security purposes. You may enable execute by doing "chmod -R +x sirikata" in cygwin, or Granting yourself Full Control in Right Click->Properties->Security (make sure to click the Replace All Entries in Child Objects checkbox).
 +
* Getting errors like "fatal error C1900: Il mismatch between 'P1' version '20080116' and 'P2' version '20070207'"? You're probably on Visual Studio Express 2008 and haven't updated to SP1.
  
 
==== DLL Path under windows ====
 
==== DLL Path under windows ====
In order to get Sirikata to find the required DLLs, you must copy all DLL files from these directories into the "build/cmake" directory:
+
For setup on Windows, run /tools/win32/copydlls.bat '''from the Sirikata root directory'''.  All that this does is copy several dependencies' dll-s (and several files from Berkelium) into the build/cmake/BUILDTYPE directory, where BUILDTYPE is Debug, Release, or RelWithDebInfo.
* dependencies\boost_1_37_0\lib
 
* dependencies\installed-curl
 
* dependencies\ogre-1.6.1\bin
 
* dependencies\SDL-1.3.0\bin
 
* dependencies\protobufs\bin
 
  
 
Note that Sirikata currently does not work in Windows 2000, due to the lack of a few raw input functions (RegisterRawInputDevice) that SDL 1.3 uses. This will hopefully be fixed in SDL at some point to make it use the old DirectX raw input system.
 
Note that Sirikata currently does not work in Windows 2000, due to the lack of a few raw input functions (RegisterRawInputDevice) that SDL 1.3 uses. This will hopefully be fixed in SDL at some point to make it use the old DirectX raw input system.
Line 176: Line 189:
  
 
= Running Sirikata =
 
= Running Sirikata =
Assuming the previous steps have completed successfully, you should have a few binaries built and you are ready to [[Running Sirikata|run Sirikata]].
+
Assuming the previous steps have completed successfully, you should have a few binaries built and you are ready to [[Getting_Started_for_Users|run Sirikata]].

Latest revision as of 19:18, 21 April 2012

Dependencies

Sirikata depends on quite a bit of external libraries. We provide install scripts and precompiled binaries to get you up and running quickly.

A makefile in the top-level directory provides commands for checking out the dependencies repository and installing them. While the depends target is usually what you want, there are a few variants you can use instead:

  • minimal-depends - the minimal dependencies required to get the system building
  • minimal-graphics-depends - the minimal dependencies required to get 3D graphics working, i.e. to run a client, but not with all the bells and whistles
  • headless-depends - all dependencies that can be run headless, i.e. everything but the graphics support. Useful to avoid building expensive dependencies and plugins when you're going to be running on a server
  • full-depends - all dependencies, enabling all features of the system, and enables use of root to install system packages

If you want more details, Dependencies gives a list of required and optional libraries and a brief description of how they are used.

Note Some dependency files are quite large, especially the precompiled binaries. When you run the install commands, it may appear the script has hung, but its likely just checking out the packages.

CMake

We use CMake on all platforms to check for dependencies and generate a build script. The Linux dependencies will install CMake from your system's package manager if you allow it to use root. Otherwise and for all Mac and Windows platforms, CMake is not installed automatically. You should use the install tool from the CMake website or install it using your system's package manager. CMake 2.6 or higher is required.

Note On OS X, when the installer asks if you want to install the command-line version of the tools, say yes. While our build system can be run through the GUI, some dependencies also use CMake and you do not want to have to perform those install steps manually.

Subversion

Because of their size, dependencies are stored in subversion repositories. If you want to use our automated dependency scripts, you'll need Subversion installed. It is installed on Mac by default, Linux users can use their distributions system package, and Windows users can either install it via Cygwin or use TortoiseSVN.

Simple Method: Makefile

On all platforms, if you have the right tools (make, subversion, and unzip on Windows provided by Cygwin) a Makefile is provided in the root directory which checks out and either builds or extracts dependencies to sirikata/dependencies/:

 cd sirikata/
 make depends

Windows Notes

The simple method requires Cygwin. All dependencies are installed locally in the Sirikata checkout. No root access is needed.

Note The Windows dependencies support Visual Studio 2008 SP1 and later, including Express editions. You must have SP1 installed for Visual Studio 2008. Visual Studio 2005 and earlier are no longer supported.

Without Cygwin (manual)

If you do not have Cygwin and do not want to install it, you can perform the steps manually. Using a subversion client, checkout

http://sirikatawin32.googlecode.com/svn/trunk/ 

to sirikata/dependencies/.

Go into the dependencies folder and unzip all of the packages directly into the dependencies directory. Make sure to select "Extract Here". If that option is not available, remove the name of the zip file from the end to indicate where to extract, so it just ends with "dependencies". The extracted directories should not have the same name as the zip files.

Then got to the top-level directory and run this command:

git submodule update --init --recursive

Mac

Required Tools

You'll need a few required tools, most of which are built in. At a minimum, you'll need git and wget. These are easy to get from homebrew.

32 vs. 64 bit

While other platforms (e.g. Linux) have supported 64 bit builds for awhile, currently many dependencies still don't support 64 bit builds on OS X. Because of this we force all builds into 32-bit i386 mode.

XCode and Developer Tools

The transitions between Leopard, Snow Leopard, and Lion, gcc 4.0 and 4.2, and XCode 3 and 4 have made it quite challenging to support all configurations of OS X development machines. Here are the configurations which are known to work and are the suggested way to configure your environment:

  • If you are on Lion, you should just install XCode 4. Currently it will include gcc 4.2.
  • If you are on Leopard or Snow Leopard you should use XCode 3.2.6 and gcc 4.0.

In both cases, when you run the dependency install script it should list whether it has found those configurations.

You can install XCode 3.2.x in parallel to XCode 4.0 if you already have that version. You can either download XCode at [1] or use the Snow Leopard DVD: under “Optional Installs”, install “Xcode.mpkg” to get 3.2. Use all default options. We highly recommend at least 3.2.2 as previous versions had issues with hanging.

Note If you already installed XCode 4, then gcc and g++ were upgraded to version 4.2. The simplest course of action is to (re)install XCode 3.2 in parallel from here -- the setup should work fine next to XCode 4. You might be able to get by just by copying the old files (gcc-4.0 and g++-4.0) from /Developer-old/usr/bin back to /Developer/usr/bin, but this probably won't work for Snow Leopard (OS X 10.6).

You may get errors of the following form: "CMake Error: The following variables are used in this project, but they are set to NOTFOUND." For the Boost_INCLUDE_DIR variable, you can edit CMakeCache.txt to set the variable to the directory containing the Boost header files. It should be something like sirikata/dependencies/boost_1_45_0/boost.

Linux

The install script expects an Ubuntu system, 8.04 or greater.

If you want to install the system package dependencies directly, issue the following command:

sudo apt-get install \
git-core cmake sed unzip zip automake1.9 nvidia-cg-toolkit jam g++ \
libzzip-dev libxt-dev libxaw7-dev libxxf86vm-dev libxrandr-dev libfreetype6-dev \
libxext-dev autoconf libtool libpcre3-dev flex bison patch libbz2-dev gawk \
libglu1-mesa-dev tofrodos freeglut3-dev scons libexpat1-dev \
libgtk2.0-dev libnss3-dev libgconf2-dev gperf libasound2-dev subversion \
libtool autoconf ruby flex libgsl0-dev libssl-dev libspeex-dev libxss-dev \
libdbus-glib-1-dev libgnome-keyring-dev libxml2-dev libjpeg62-dev libcups2-dev \
wget libcurl4-gnutls-dev build-essential libxmu-dev libxi-dev libusb-1.0-0-dev \
yasm

For Ubuntu 11.04 and earlier, add:

sudo apt-get install libglut3-dev

These packages should cover everything you need for the entire system, including the graphical client and all scripting plugins.

Building Sirikata

We use CMake to generate our build scripts. Make is used on Mac and Linux and Visual Studio is used on Windows to perform the actual build. All three builds follow the same basic steps:

  • Run cmake, possibly modifying the configuration.
  • Run your build tool.

Windows

Start up CMake and point both paths to sirikata/build/cmake. Hit configure twice. If you installed any dependencies in non-standard locations, point CMake to them now. To generate the build files, hit Generate. On Windows Vista or later you might want to change CMAKE_INSTALL_PREFIX to the directory where you want to install your compiled binaries and hit configure again, because by default CMake will generate Visual Studio project that will try to install compiled binaries into %PROGRAMFILES%. This requires administrative privileges, which is why build will fail (unless you run Visual Studio with administrative privileges or have disabled UAC).

Now browse to sirikata/build/cmake. Open Sirikata.sln. Select the configuration you want to build -- we highly recommend RelWithDebInfo because Debug runs very slowly and RelWithDebInfo still gets you useful information when run in the debugger. Finally, run Build All.

This should result in all libraries, plugins, and binaries in the sirikata/build/cmake/debug or sirikata/build/cmake release, depending on which configuration you built.

Notes

  • There aren't standard locations to search for dependencies on Windows. Obviously we've setup the build system to work cleanly with the dependency install script. The easiest way to get a manual installation of dependencies on Windows to work is to use the same layout as the install script, where all dependencies are located in sirikata/dependencies. If you don't do this, you will almost certainly need to manually specify the locations of some libraries in CMake.
  • The build is known to work for VS2008, for both regular and Express versions.
  • If you get error 0xc0000022, check the permissions of the sirikata top-level directory. If you used Cygwin's version of git, it will remove execute permissions for security purposes. You may enable execute by doing "chmod -R +x sirikata" in cygwin, or Granting yourself Full Control in Right Click->Properties->Security (make sure to click the Replace All Entries in Child Objects checkbox).
  • Getting errors like "fatal error C1900: Il mismatch between 'P1' version '20080116' and 'P2' version '20070207'"? You're probably on Visual Studio Express 2008 and haven't updated to SP1.

DLL Path under windows

For setup on Windows, run /tools/win32/copydlls.bat from the Sirikata root directory. All that this does is copy several dependencies' dll-s (and several files from Berkelium) into the build/cmake/BUILDTYPE directory, where BUILDTYPE is Debug, Release, or RelWithDebInfo.

Note that Sirikata currently does not work in Windows 2000, due to the lack of a few raw input functions (RegisterRawInputDevice) that SDL 1.3 uses. This will hopefully be fixed in SDL at some point to make it use the old DirectX raw input system.

Mac and Linux

For convenience we provide a top level makefile which performs the standard build operations. If you want a default build and have used the install script for dependencies, do:

 cd sirikata/
 make

If you'd like to run the build manually, do the following (essentially what is in the makefile):

 cd sirikata/build/cmake
 cmake . [-DCMAKE_BUILD_TYPE=Debug|Release]
 make

To interactively adjust settings, for instance to point CMake to a different version of libraries:

 cd sirikata/build/cmake
 ccmake .

When the build completes you should have the libraries, plugins, and binaries in sirikata/build/cmake.

Additional Tools

If you're planning on developing Sirikata, it can be helpful to use ccache or a distributed compiler tool like distcc. These tools speed up compilation, either avoiding duplicate work after running make clean or by distributed compile jobs across the network. We provide a wrapper script that tries to use these automatically. Instead of the steps above, use this script:

 cd sirikata/build/cmake
 rm CMakeCache.txt   # We need to clear out old settings if you already ran CMake
 ./cmake_with_tools.sh .

The script passes other settings along to CMake, so you can add any parameters you would normally pass to CMake to this command as well.

Mac XCode Project

Many mac developers prefer to use the XCode programming environment. To do this, cmake will need to be configured to use XCode

Steps are:

  1. test -d build/cmake/Debug || mkdir build/cmake/Debug
        1. The directory for XCode build products.
  2. python csv_converter.py build/cmake/Debug/scene.csv scene.db
        1. Generate a scene for use by the XCode project.
  3. cd build/cmake
  4. ln -s ../../dependencies/Frameworks Frameworks
        1. To fix brittle paths in Xcode project.
  5. cmake -G Xcode .
  6. cd Debug
  7. ln -s ../Sirikata.Protocol.dll Sirikata.Protocol.dll
  8. ln -s ../Sirikata.Runtime.dll Sirikata.Runtime.dll
  9. cd ..
  10. Launch Xcode
  11. Open Sirikata.xcodeproj
  12. Edit project settings
        1. General tab
        2. Choose project root
        3. Select build/cmake directory. Click OK. Value should just be <Project File Directory>.
              1. This fixes a problem that prevents debugger from working.
  13. Project menu
        1. Select active target ALL_BUILD
        2. Select active project executable cppoh
        3. Select active build configuration Debug
  14. Close XCode to save the project
  15. Reopen Sirikata.xcodeproj
  16. Build it
  17. Debugger
        1. Confirm that you can set a break point.
        2. Run debugger
        3. Confirm that it stops at a break point.
        4. Confirm that it displays source code.
        5. (trouble shoot back to step 10)

Running Sirikata

Assuming the previous steps have completed successfully, you should have a few binaries built and you are ready to run Sirikata.