- MySQL clustering strategies and comparisions
- Howto MySQL DRBD HA
- Howto GFS GNBD
- Howto GFS GNBD(CentOS)
- LVS/NAT groundwork
SVN STACK Design / Install / Configure / Repository Mirroring(write thru proxy) / Repository Migration(cvs2svn), Repository Browsing(websvn), Repository Search(gonzui) - II
Continued from SVN STACK Design /
Install / Configure / Repository Mirroring(write thru proxy) /
Repository Migration(cvs2svn), Repository Browsing(websvn), Repository
Search(gonzui) - I
5. SVN Install Instructions( continued from previous article)
* SVNSYNC
* REVERSE SYNC
* SYNC AUTOMATE
* WRITE THRU PROXY
6. Errors & Resolutions
7. WebSVN
8. gonzui
SVNSYNC
On MySubversionAmericaServer:
/home/myname/apache-2.2.6/bin/htpasswd -m /home/myname/apache-2.2.6/etc/svn-auth-file svnsync
Password: mylab
/etc/rc3.d/S99svn reload
TODO: remove svn rpm default installation
su svn
/home/myname/subversion/bin/svnadmin create /svnroot/MySubversionIndiaServer
cp /svnroot/MySubversionIndiaServer/hooks/start-commit.tmpl /svnroot/MySubversionIndiaServer/hooks/start-commit
vi /svnroot/MySubversionIndiaServer/hooks/start-commit
:50 Added below lines
if [ "$USER" = "svnsync" ]; then exit 0; fi
echo "Only the svnsync user may commit new revisions as this is a read-only, mirror repository." >&2
exit 1
cp /svnroot/MySubversionIndiaServer/hooks/pre-revprop-change.tmpl
/svnroot/MySubversionIndiaServer/hooks/pre-revprop-change
vi /svnroot/MySubversionIndiaServer/hooks/pre-revprop-change
:63 Added below lines
if [ "$USER" = "svnsync" ]; then exit 0; fi
echo "Only the svnsync user may change revision properties as this is a
read-only, mirror repository." >&2
exit 1
chmod -R 0755 /svnroot/MySubversionIndiaServer/hooks/*
vi /home/myname/apache-2.2.6/conf/httpd.conf
#Extra section added for MySubversionIndiaServer
#Subversion repository
<Location /MySubversionIndiaServer>
DAV svn
SVNPath /svnroot/MySubversionIndiaServer
#SVNAutoversioning on
#ModMimeUsePathInfo on
AuthType Basic
AuthName "Subversion repository"
AuthUserFile /home/myname/apache-2.2.6/etc/svn-auth-file
Require valid-user
</Location>
chown -R svn:svn /svnroot/MySubversionIndiaServer
/etc/rc3.d/S99svn reload
On MySubversionIndiaServer:
/home/myname/svn/softwares/apache-2.2.6/bin/htpasswd -m /home/myname/svn/softwares/apache-2.2.6/etc/svn-auth-file svnsync
Password: <PASSWORD>
/etc/init.d/httpd-svn reload
svnsync initialize
http://MySubversionAmericaServer/MySubversionIndiaServer/
http://MySubversionIndiaServer/svn/ --username=svnsync
--password=<PASSWORD>
svnsync synchronize
http://MySubversionAmericaServer/MySubversionIndiaServer/
--username=svnsync --password=<PASSWORD>
REVERSE SYNC:
On MySubversionIndiaServer:
su svn
svnadmin create /home/myname/svn/MySubversionAmericaServer
vi /home/myname/svn/MySubversionAmericaServer/hooks/start-commit.tmpl
:50 Added below lines
if [ "$USER" = "svnsync" ]; then exit 0; fi
echo "Only the svnsync user may commit new revisions as this is a read-only, mirror repository." >&2
exit 1
cp /home/myname/svn/MySubversionAmericaServer/hooks/start-commit.tmpl
/home/myname/svn/MySubversionAmericaServer/hooks/start-commit
vi /home/myname/svn/MySubversionAmericaServer/hooks/pre-revprop-change.tmpl
:63 Added below lines
if [ "$USER" = "svnsync" ]; then exit 0; fi
echo "Only the svnsync user may change revision properties as this is a
read-only, mirror repository." >&2
exit 1
cp
/home/myname/svn/MySubversionAmericaServer/hooks/pre-revprop-change.tmpl
/home/myname/svn/MySubversionAmericaServer/hooks/pre-revprop-change
chmod -R 0755 /home/myname/svn/MySubversionAmericaServer/hooks/*
vi /home/myname/svn/softwares/apache-2.2.6/conf/httpd.conf
#Extra section added for MySubversionIndiaServer
#Subversion repository
<Location /MySubversionAmericaServer>
DAV svn
SVNPath /home/myname/svn/MySubversionAmericaServer
#SVNAutoversioning on
#ModMimeUsePathInfo on
AuthType Basic
AuthName "Subversion repository"
AuthUserFile /home/myname/svn/softwares/apache-2.2.6/etc/svn-auth-file
Require valid-user
</Location>
chown -R svn:svn /home/myname/svn/MySubversionAmericaServer
/etc/init.d/httpd-svn reload
On MySubversionAmericaServer:
/home/myname/subversion/bin/svnsync initialize
http://MySubversionIndiaServer/MySubversionAmericaServer/
http://MySubversionAmericaServer/svn/ --username=svnsync
--password=<PASSWORD>
/home/myname/subversion/bin/svnsync synchronize
http://MySubversionIndiaServer/MySubversionAmericaServer/
--username=svnsync --password=<PASSWORD>
Error:
svnsync: PROPFIND of '/MySubversionAmericaServer': Could not resolve hostname `MySubversionIndiaServer'
Resolution:
ip instead of hostname
/home/myname/subversion/bin/svnsync initialize
http://MySubversionIndiaServer/MySubversionAmericaServer/
http://MySubversionAmericaServer/svn/ --username=svnsync
--password=<PASSWORD>
/home/myname/subversion/bin/svnsync synchronize
http://MySubversionIndiaServer/MySubversionAmericaServer/
--username=svnsync --password=<PASSWORD>
SYNC AUTOMATE:(using post commit hooks, but, not crons)
MySubversionAmericaServer:
su svn
cp /svnroot/training/hooks/post-commit.tmpl /svnroot/training/hooks/post-commit
vi /svnroot/training/hooks/post-commit
Added at the end of the file:
/home/myname/subversion/bin/svnsync synchronize
http://MySubversionIndiaServer/MySubversionAmericaServer/
--username=svnsync --password=<PASSWORD>
exit 0
chmod 0755 /svnroot/training/hooks/*
MySubversionIndiaServer:
su svn
cp /home/myname/svn/projects/hooks/post-commit.tmpl /home/myname/svn/projects/hooks/post-commit
vi /home/myname/svn/projects/hooks/post-commit
Added at the end of the file:
/home/myname/svn/softwares/subversion-1.4.5/bin/svnsync synchronize
http://MySubversionAmericaServer/MySubversionIndiaServer/
--username=svnsync --password=<PASSWORD>
exit 0
chmod 0755 /home/myname/svn/projects/hooks/*
Test:
modify a file in MySubversionIndiaServer and browse it at MySubversionAmericaServer - OK
modify a file in MySubversionAmericaServer and browse it at MySubversionIndiaServer - OK
ALTERNATIVELY, SYNC CAN BE AUTOMATED USING CRON AND REMOVING POST COMMIT HOOK
Howto co from mirror and commit to master:
Plan 1) (DROPPED)
To get the UUID on the master machine:
svnadmin dump -r0 /home/myname/svn/projects | head -n 3 > saved-uuid
Copy the file saved-uuid to the mirror machine and do a
svnadmin load --force-uuid /svnroot/MySubversionIndiaServer < saved-uuid
On client:
svn switch --relocate
svn commit
Plan 2) (IMPLEMENTED) WRITE THRU PROXY
Use Apache(mod_proxy) and SVNMasterURI(available since SVN beta1)
In MySubversionIndiaServer:
Install & enable mod_proxy:
cd /home/myname/svn/softwares/apache-2.2.6/bin
./apxs -c -i
/home/myname/svn-install-scripts/httpd-2.2.6/modules/proxy/mod_proxy.c
/home/myname/svn-install-scripts/httpd-2.2.6/modules/proxy/proxy_util.c
./apxs -c -i /home/myname/svn-install-scripts/httpd-2.2.6/modules/proxy/mod_proxy_http.c
vi /home/myname/svn/softwares/apache-2.2.6/conf/httpd.conf
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
vi /home/myname/svn/softwares/apache-2.2.6/conf/httpd.conf
#SVN 1.5 beta india
<Location /svn>
DAV svn
SVNPath /home/myname/svn/testproject
#SVNAutoversioning on
#ModMimeUsePathInfo on
AuthType Basic
AuthName "Subversion repository"
AuthUserFile
/home/myname/svn/softwares/apache-2.2.6/etc/svn-auth-file
Require valid-user
</Location>
#SVN 1.5 beta us for sync only
<Location /MySubversionAmericaServer>
DAV svn
SVNPath /home/myname/svn/MySubversionAmericaServer
Order deny,allow
Deny from all
Allow from MySubversionAmericaServer # or provide IP
#SVNAutoversioning on
#ModMimeUsePathInfo on
AuthType Basic
AuthName "Subversion repository"
AuthUserFile /home/myname/svn/softwares/apache-2.2.6/etc/svn-auth-file
Require valid-user
</Location>
#SVN 1.5 beta us synced for co only
<Location /CO-from-INDIA-CIN-to-AMERICA/>
DAV svn
SVNPath /home/myname/svn/MySubversionAmericaServer
SVNMasterURI http://MySubversionAmericaServer/svn/
#SVNAutoversioning on
#ModMimeUsePathInfo on
AuthType Basic
AuthName "Subversion repository"
AuthUserFile /home/myname/svn/softwares/apache-2.2.6/etc/svn-auth-file
Require valid-user
</Location>
In MySubversionAmericaServer:
cd /home/myname/apache-2.2.6/bin
./apxs -c -i
/home/myname/svn-install-scripts/httpd-2.2.6/modules/proxy/mod_proxy.c
/home/myname/svn-install-scripts/httpd-2.2.6/modules/proxy/proxy_util.c
./apxs -c -i /home/myname/svn-install-scripts/httpd-2.2.6/modules/proxy/mod_proxy_http.c
vi /home/myname/apache-2.2.6/conf/httpd.conf
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
vi /home/myname/apache-2.2.6/conf/httpd.conf
#SVN 1.5 beta us
<Location /svn>
DAV svn
SVNPath /svnroot/testproject
#SVNAutoversioning on
#ModMimeUsePathInfo on
AuthType Basic
AuthName "Subversion repository"
AuthUserFile /home/myname/apache-2.2.6/etc/svn-auth-file
Require valid-user
</Location>
#SVN 1.5 beta india for sync only
<Location /MySubversionIndiaServer>
DAV svn
SVNPath /svnroot/MySubversionIndiaServer
Order deny,allow
Deny from all
Allow from MySubversionIndiaServer # or provide IP
#SVNAutoversioning on
#ModMimeUsePathInfo on
AuthType Basic
AuthName "Subversion repository"
AuthUserFile /home/myname/apache-2.2.6/etc/svn-auth-file
Require valid-user
</Location>
#SVN 1.5 beta synced for co only
<Location /CO-from-AMERICA-CIN-to-INDIA/>
DAV svn
SVNPath /svnroot/MySubversionIndiaServer
SVNMasterURI http://MySubversionIndiaServer/svn/
#SVNAutoversioning on
#ModMimeUsePathInfo on
AuthType Basic
AuthName "Subversion repository"
AuthUserFile /home/myname/apache-2.2.6/etc/svn-auth-file
Require valid-user
</Location>
Errors & Resolutions
Install errors and resolutions not covered here.
1) During svnsync usage
Error:
svnsync: PROPFIND of '/MySubversionAmericaServer': Could not resolve hostname `MySubversionIndiaServer'
Resolution:
ip instead of hostname
2)
SVNSYNC troubleshooting:
Trouble shooting trailing slases in httpd.conf
I tried Subversion 1.5 Beta1 WebDAV Write-Thru Proxies. Works fine for me.
While compiling SVN 1.5, it complained about few thing(mostly related to Neon
and Swig-Python).. which were resolved after I provided it with latest version of Neon and Swig.
I tried a mirrored(svnsync) setup like:
A -------------> a
b <------------- B
where A,B are masters and a,b are slaves. Later I tried
checkout-from-mirror-checkin-to-master scenario.
Works well. It gave me trailing slash related error.
Resolution:
In master httpd.conf no trailing slash in location directive:
<Location /ABCD>
In mirror httpd.conf has trailing slash in location directive:
<Location /co-from-mirror-cin-to-master/>
SVNMasterURI http://masterhost/ABCD/
..
..
</Location>
Complication of the above setup should be noted first.
On checkout from mirror and checkin to master , this setup does the following.
1) redirect to master
2) cin to master
3) sync to mirror (using svnsync in post-commit wrapper)
Setup involves svnsync, location directive, SVNMasterURI. So this whole slash
thing observation may not be a actual problem at all. I may have gone wrong
somewhere. Anyways.. as I said.. it was resolved with no further problem.
master: noslash
mirror: slash
SVNMasterURI: slash
=OK
master: noslash
mirror: slash
SVNMasterURI: noslash
=svn: Unusable URI: it does not refer to this repository
master: noslash
mirror: noslash
SVNMasterURI: slash
=svn: Server sent unexpected return value (405 Method Not Allowed) in response
to PROPPATCH request for
'/co-from-mirror-cin-to-master!svn/wbl/1e1fafe4-c5f4-48ba-93c3-05646e5b9087/3
master: slash
mirror: slash
SVNMasterURI: slash
=svnsync: Server sent unexpected return value (405 Method Not Allowed) in
response to PROPFIND request for '/ABCD'
3) During cvsus --> MySubversionAmericaServer conversion using cvs2svn
Error:
'/home/myname/cvs/cvsroot/dummy/dummy.txt,v' is not a valid ,v file
Resolution:
It is an empty file(not required) and can be removed during conversion
Checkout from mycvsserver says:
cvs -d :pserver:praveen@mycvsserver:2401/cvsroot co dummy/dummy.txt
cvs server: warning: new-born dummy/dummy.txt
has disappeared
cvs -d :pserver:praveen@mycvsserver:2401/cvsroot co dummy
cvs server: Updating
dummy
U
dummy/dummy.txt
mv /home/myname/cvs/cvsroot/dummy/dummy.txt,v /tmp/junk/
Error:
It is not clear how the following symbols should be converted.
Use --force-tag, --force-branch, --exclude, and/or --symbol-default to
resolve the ambiguity.
'ABCDEF' is a tag in 11111 files, a branch in 111
files and has commits in 222 files
'GEFGEF' is a tag in 333 files, a branch in 1111 files
and has commits in 11 files
Temporary Resolution:
--symbol-default="heuristic"
Error:
Unable to convert a path
'whatever.whatever'
to internal encoding.
Consider rerunning with one or more '--encoding' parameters or
with '--fallback-encoding'.
Resolution:
--encoding=latin_1
--encoding=utf_8
--encoding=utf_7
--encoding=utf_16
--encoding=shift_jis
--fallback-encoding=utf_8
4) After installing svn-python binding in MySubversionAmericaServer.. while executing
python
>>>import svn.repos
Error:
undefined symbol: gss_delete_sec_context
Resolution:
cd /home/myname/svn-install-scripts/subversion-1.5.0-beta1
vi Makefile
#SVN_APR_LIBS = /home/myname/apache-2.2.6/lib/libapr-1.la -luuid -lrt -lcrypt -lpthread -ldl
SVN_APR_LIBS = /home/myname/apache-2.2.6/lib/libapr-1.la -luuid -lrt -lcrypt -lpthread -ldl -lgssapi
ln -s /usr/lib/libgssapi.so.2.0.0 /usr/lib/libgssapi.so
And make again.
WebSVN-2.0:
Requires Apache with php-4.3+
Hence use RHEL default httpd+php installation
Verify php -v
Install,
cd /home/myname/svn/
svn --username=guest co http://websvn.tigris.org/svn/websvn/tags/2.0/ websvn-2.0
Password: Hit enter
cd websvn-2.0
Edited httpd.conf and add proper host:port
<VirtualHost MySubversionIndiaServer:8080>
ServerName MySubversionIndiaServer
DocumentRoot /home/myname/svn/websvn-2.0
SetEnv HTTP_PORT 8080
ErrorLog /var/log/httpd/error_log.websvn
</VirtualHost>
Do an Apache graceful
/etc/init.d/httpd-new graceful
Browse http://MySubversionIndiaServer/doc/install.html
chmod -R 0755 /home/myname/svn/websvn-2.0/cache
cd /home/myname/svn/websvn-2.0/
mv include/distconfig.php include/config.php
Edit include/config.php
Browse http://MySubversionIndiaServer:8080/
gonzui:
Install Ruby
echo "Installing Ruby:"
cd /home/myname/svn-install-scripts/
#wget ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.6.tar.gz
tar zxvf ruby-1.8.6.tar.gz
cd ruby-1.8.6
CFLAGS="-m32" ./configure
make
make install
Install BDB
echo "Installing bdb:"
cd /home/myname/svn-install-scripts/
#wget ftp://moulon.inra.fr/pub/ruby/bdb.tar.gz
tar zxvf bdb.tar.gz
cd bdb-0.6.2
make distclean
make clean
ruby extconf.rb
make
make install
Installing Gonzui
echo "Installing gonzui:"
cd /home/myname/svn-install-scripts/
#wget http://nchc.dl.sourceforge.net/sourceforge/gonzui/gonzui-1.1.tar.gz
tar zxvf gonzui-1.1.tar.gz
cd gonzui-1.1
If its a 32bit arch node:
CFLAGS="-m32"
./configure
--prefix=/home/myname/svn/softwares/gonzui
If its a 64bit arch node:
CFLAGS="-m64 -fPIC"
./configure
--prefix=/home/myname/svn/softwares/gonzui
make
make install
Gonzui post installation
cd /home/myname/svn/softwares/gonzui/
cp etc/gonzuirc.sample ~/.gonzuirc
Edit ~/.gonzuirc
:daemon => true,
:http_port => 8081,
:site_title => "My Repository - Code Search",
TO ADD CVS REPOS:
cd /home/myname/svn/softwares/gonzui/bin
cvs -d :pserver:praveen@mycvsserver:/cvsroot login
Example:
./gonzui-import --cvs :pserver:praveen@mycvsserver:/cvsroot dummyproject
TO ADD SVN REPOS:
cd /home/myname/svn/softwares/gonzui/
./gonzui-import --svn http://MySubversionIndiaServer/svn/dummyproject dummyproject
Run gonzui server
./gonzui-server
Run svn updates to gonzui
./gonzui-update
List whats in gonzui
./gonzui-db --list
If added content is to be removed:
./gonzui-remove <>
Set a cron to update repository changes to gonzui
crontab -e
*/1 * * * * /home/myname/svn/softwares/gonzui/bin/gonzui-update
Add startup. Edit /etc/rc.local
/home/myname/svn/softwares/gonzui/bin/gonzui-server
SVN STACK Design / Install / Configure / Repository Mirroring(write thru proxy) / Repository Migration(cvs2svn), Repository Browsing(websvn), Repository Search(gonzui) - I
1. SVN What?
2. SVN Why?
3. SVN How?
4. SVN Stack
5. SVN Install Instructions
* Apache for SVN
* Subversion
* Subversion authentication & authorization
* cvs2svn
SVN What?
In computing, Subversion (SVN) is a version control system initiated in 2000 by CollabNet Inc. It is
used to maintain current and historical versions of files such as source code, web pages, and documentation.
Its goal was to be a mostly-compatible successor to the widely used Concurrent Versions System. The software
is released under an Apache/BSD-style open source license.
Subversion is well-known in the open source community and is used by many open source projects such as:
Apache Software Foundation, KDE, GNOME, Free Pascal, GCC, Python, Ruby, Samba and Mono. SourceForge.net and
Tigris.org also provide Subversion hosting for their open source projects. Google Code and BountySource
systems use it exclusively.
In a 2007 report by Forrester Research, Subversion was recognized as the sole leader in the Standalone
Software Configuration Management (SCM) category and strong performer in the Software Configuration and
Change Management (SCCM) category.
Versions I tried:
Stable : 1.4.5
Beta : 1.5.0-beta1
SVN Why?
Features:
* Most current CVS features.
* Directories, renames, and file meta-data are versioned. Versioning of symbolic links.
* Commits are truly atomic. Interrupted commit operations do not cause repositoryinconsistency or corruption.
* Apache network server option, with WebDAV/DeltaV protocol, Standalone option.
* Branching and tagging are cheap (constant time) operations.
In Subversion the individual files do not carry version numbers. Instead,
versioning is applied to the entire repository, and this version number is
bumped up on every commit. As a result, a tag in Subversion is simply the
version number or release number at the time of the snapshot. Since worded
names tend to be more memorable than numeric names, Subversion administrators
typically use a place metaphor, and create tags by copying the project to
a named subdirectory of tags. This is accomplished using svn copy.
* Code is well expandable, and future improvements are coming. Natively client/server, layered library design. CVS lacks this.
* Client/server protocol sends diffs in both directions. Offline diffs, and they're instant.
* Language bindings like Ruby, Python, etc.
* Native support for binary files, with space-efficient binary-diff storage.
* Repository mirroring.
* Choice of database or plain-file repository implementations.
* Easier hook scripts.
* Parsable output, including XML log output.
* Internationalization.
* Full MIME support.
* Lots of Subversion support projects
Eg:
svn2cvs - save subversion commits to (read-only) cvs repository
SubTrain - open source trainings for Subversion
TortoiseSVN - a client
svn4office - integrating Subversion in MS Office software
SvnMapper - gives you a map to navigate your subversion branches
Merge tracking client
* In 1.5.0-beta1:
Merge tracking (foundational).
WebDAV transparent write-through proxy.
Interactive conflict resolution.
Sparse checkouts (via new --depth option).
And more.
SVN v/s CVS links:
http://en.wikipedia.org/wiki/Comparison_of_revision_control_software
http://wiki.scummvm.org/index.php/CVS_vs_SVN
http://www.bioperl.org/wiki/Why_SVN
http://www.devmaster.net/forums/showthread.php?t=7394
http://www.pushok.com/soft_svn_vscvs.php (old link)
http://svnbook.red-bean.com/en/1.1/apa.html
SVN How?
What we require to migrate from cvs?
cvs2svn
A CVS to Subversion conversion program
How to serve repository?
Locally,
It can be accessed locally or over network share (file://)
Remotely,
Using svnserve protocol (svn://) - fast, no logging
Using svnserve over SSH (svn+ssh://) - fast, no logging
Using Apache WebDAV (http://) - bit slow, logs, many auth methods, beats corporate firewalls
What are required additional resources?
Repository mirroring:
svnsync
Svnsync is a tool for creating and maintaining read-only mirrors of
subversion repositories. It works by replaying commits that occurred
in one repository and committing it into another. Similar to CVSup.
SVN::Notify::Mirror::Rsync
Mirror a repository path via Rsync
Refer http://search.cpan.org/~jpeacock/SVN-Notify-Mirror-0.03603/lib/SVN/Notify/Mirror/Rsync.pm
Repository browsing:
WebSVN
Similar to CVSweb
ViewVC
Similar to CVSweb
Apache(WebDAV)
Built-in repository browsing on Apache
gonzui
A very good source code search tool which looks like google search
It can search repositories, tar-balls, etc
StatSVN
Repository statistics
What clients are to be tried?
tortoisesvn
A Subversion client, implemented as a windows shell extension
subclipse
Subclipse is an Eclipse Team Provider plug-in providing support for Subversion within the Eclipse IDE
RapidSVN
RapidSVN is a cross-platform GUI front-end for the Subversion revision system
SVN Stack
httpd-2.2.6 [Apache/2.2.6 (Unix) mod_ssl/2.2.6 OpenSSL/0.9.7a DAV/2 SVN/1.5.0-beta1 with mod_proxy]
Dependencies
|__svn
|__openssl
|__zlib
Provides
|__httpd,apxs,apr-1-config,apachectl,htpasswd,etc
subversion-1.5.0-beta1
Dependencies
|__apache(apxs,apr-util)
|__jdk
|__openssl
|__zlib
|__neon-0.28.1
|__swig-1.3.34
Provides
|__svn,svnadmin,svndumpfilter,svnlook,svnserve,svnsync,etc
cvs2svn-2.0.1
Dependencies
|__python
Provides
|__cvs2svn
gonzui-1.1
Dependencies
|__bdb-0.6.2
|__ruby-1.8.6
Provides
|__gonzui-server(repository indexed search server),gonzui-import,gonzui-update,gonzui-db,etc
websvn-2.0
Dependencies
|__svn
|__apache
|__php-4.3+
Provides
|__repository browsing
SVN Install Instructions
Apache for SVN:
Now,
mkdir /home/myname/svn-install-scripts
cd /home/myname/svn-install-scripts/
wget http://mirrors.24-7-solutions.net/pub/apache/httpd/httpd-2.2.6.tar.gz
gunzip httpd-2.2.6.tar.gz
tar xvf httpd-2.2.6.tar
cd httpd-2.2.6
If 32 bit arch use "-m32" in CFLAGS, elseif 64 bit arch use "-m64 -fPIC" in arch.
CFLAGS="-m32"
./configure
--prefix=/home/myname/svn/softwares/apache-2.2.6
--enable-rewrite=shared
--with-z=/usr
--enable-cgi=shared
--enable-info=shared
--enable-ssl=shared
--enable-so
--enable-ext-filter=shared
--enable-log-forensic=shared
--enable-mime-magic=shared
--enable-expires=shared
--enable-usertrack=shared
--enable-headers=shared
--enable-unique-id=shared
--enable-dav=shared
--enable-dav-fs=shared
--enable-vhost-alias=shared
--with-mpm=prefork
--enable-nonportable-atomics=yes
make
make install
Edit /home/myname/svn/softwares/apache-2.2.6/conf/httpd.conf
Listen 80
#OR SOME CUSTOM PORT
#Selinux should be permissive or disabled if you require to run in any port less than 1000
POST INSTALL CONFIGURATION AND LSB SCRIPTS:
Only if ipaddress, hostname is found missing in /etc/hosts.. add it.
Eg:
echo "MySubversionIndiaServer MySubversionIndiaServer.mylab.com MySubversionIndiaServer" >> /etc/hosts
Create & Edit /etc/rc.d/init.d/httpd-svn
#SET APPROPRIATE PATHS - REPLACE APACHE HOME BY THE PREFIX SUPPILED DURING APACHE INSTALL
chmod 0755 /etc/rc.d/init.d/httpd-svn
chkconfig --add httpd-svn
chkconfig --level 2345 httpd-svn on
/etc/rc.d/init.d/httpd-svn start
Subversion:
To install,
cd /home/myname/svn-install-scripts/
You can either download svn package thru http or checkout from svn
#wget http://subversion.tigris.org/downloads/subversion-1.4.5.tar.gz
wget http://subversion.tigris.org/downloads/subversion-<YOUR VERSION>.tar.gz
svn co http://svn.collab.net/repos/svn/tags/1.5.0-beta1 subversion-1.5.0-beta1
cd subversion-1.5.0-beta1
To install,
groupadd svn
useradd -g svn svn
sh autogen.sh
If 32 bit arch use "-m32" in CFLAGS, elseif 64 bit arch use "-m64 -fPIC" in arch.
NOTE: compile fails if you don't have dependencies pre-installed. See DEBUG TIPS below.
Locate and take note of Apache/apxs/jdk/neon/swig/zlib/ssl paths and replace them below
CFLAGS="-m32" ./configure
--prefix=/home/myname/svn/softwares/subversion-1.5.0-beta1
--with-apxs=/home/myname/svn/softwares/apache-2.2.6/bin/apxs
--with-apr=/home/myname/svn/softwares/apache-2.2.6
--with-apr-util=/home/myname/svn/softwares/apache-2.2.6
--with-jdk=/home/myname/jdk1.3.1_09/include
--with-zlib=/usr
--with-ssl
--with-neon=/home/myname/svn/softwares/neon-0.28.1
--with-swig=/home/myname/svn/softwares/swig-1.3.34/bin/swig
#OR# --with-swig=/usr/bin/swig
make
make install
make swig-py
make install-swig-py
DEBUG TIPS:
An appropriate version of neon could not be found, so libsvn_ra_neon
will not be built. If you want to build libsvn_ra_neon, please either
install neon 0.28.1 on this system
or
get neon 0.28.1 from:
http://www.webdav.org/neon/neon-0.28.1.tar.gz
unpack the archive using tar/gunzip and rename the resulting
directory from ./neon-0.28.1/ to ./neon/
cd /home/myname/svn-install-scripts
wget http://www.webdav.org/neon/neon-0.28.1.tar.gz
tar zxvf neon-0.28.1.tar.gz
mv neon-0.28.1/ neon/
cd neon
CFLAGS="-m32" ./configure --prefix=/home/myname/svn/softwares/neon-0.28.1
make
make install
cd /home/myname/svn/softwares/neon-0.28.1/
cd /home/myname/svn-install-scripts
wget http://jaist.dl.sourceforge.net/sourceforge/swig/swig-1.3.34.tar.gz
tar zxvf swig-1.3.34.tar.gz
cd swig-1.3.34
CFLAGS="-m32" ./configure --prefix=/home/myname/svn/softwares/swig-1.3.34
make
make install
After this you can find 2 Apache modules installed. Also httpd.conf will be changed accordingly.
ls /home/myname/svn/softwares/apache-2.2.6/modules/ | grep svn
You should see
mod_authz_svn.so
mod_dav_svn.so
Install subversion python binding
If 32 bit arch:
echo /home/myname/svn/softwares/subversion-1.5.0-beta1/lib/svn-python > /usr/lib/python2.3/site-packages/subversion.pth
ln -s /home/myname/svn/softwares/subversion-1.5.0-beta1/lib/libsvn* /usr/lib/
cd /usr/lib/python*/site-packages
If 64 bit arch:
echo /home/myname/svn/softwares/subversion-1.5.0-beta1/lib/svn-python > /usr/lib/python2.3/site-packages/subversion.pth
ln -s /home/myname/svn/softwares/subversion-1.5.0-beta1/lib/libsvn* /usr/lib/
echo /home/myname/svn/softwares/subversion-1.5.0-beta1/lib/svn-python > /usr/lib64/python2.3/site-packages/subversion.pth
ln -s /home/myname/svn/softwares/subversion-1.5.0-beta1/lib/libsvn* /usr/lib64/
cd /usr/lib64/python*/site-packages
ln -s /home/myname/svn/softwares/subversion-1.5.0-beta1/lib/svn-python/svn ./svn
ln -s /home/myname/svn/softwares/subversion-1.5.0-beta1/lib/svn-python/libsvn ./libsvn
python
In python prompt >>> below type: "import svn.repos" and press enter. Later press Ctrl-D.
Set paths and permissions
chown -R svn:svn /home/myname/svn/softwares/subversion-1.5.0-beta1
echo "export PATH=${PATH}:/home/myname/svn/softwares/subversion-1.5.0-beta1/bin" >> /root/.bashrc
source /root/.bashrc
Now, to serve repository thru Apache
echo "
<Location /svn>
DAV svn
SVNPath /home/myname/svn/projects
#SVNAutoversioning on
#ModMimeUsePathInfo on
</Location>
" >> /home/myname/svn/softwares/apache-2.2.6/conf/httpd.conf
Edit /home/myname/svn/softwares/apache-2.2.6/conf/httpd.conf
User svn
Group svn
Create sample repository
svnadmin create /home/myname/svn/projects
mkdir -p /home/myname/svn/wc_projects/dummyproject
mkdir /home/myname/svn/wc_projects/dummyproject/trunk
mkdir /home/myname/svn/wc_projects/dummyproject/tags
mkdir /home/myname/svn/wc_projects/dummyproject/branch
chown -R svn:svn /home/myname/svn/projects
su svn
svn import /home/myname/svn/wc_projects file:///home/myname/svn/projects -m "Initial project structure import"
exit
cd /home/myname/svn
mv wc_projects wc_projects_old
chown svn:svn /home/myname/svn
After an Apache graceful we can access the repo as http://<IPADDRESS>:<PORT>/svn/projects
/etc/rc.d/init.d/httpd-svn graceful
Try check out
su svn
svn co http://MySubversionIndiaServer/svn wc_projects
Subversion authentication & authorization:
Apache based authentication
mkdir /home/myname/svn/softwares/apache-2.2.6/etc
Create username & passwd
/home/myname/svn/softwares/apache-2.2.6/bin/htpasswd -cm /home/myname/svn/softwares/apache-2.2.6/etc/svn-auth-file admin
/home/myname/svn/softwares/apache-2.2.6/bin/htpasswd /home/myname/svn/softwares/apache-2.2.6/etc/svn-auth-file praveen
Edit /home/myname/svn/softwares/apache-2.2.6/conf/httpd.conf
<Location /svn>
DAV svn
SVNPath /home/myname/svn/projects
#SVNAutoversioning on
#ModMimeUsePathInfo on
AuthType Basic
AuthName "Subversion repository"
AuthUserFile /home/myname/svn/softwares/apache-2.2.6/etc/svn-auth-file
Require valid-user
</Location>
Do an Apache graceful
/etc/rc.d/init.d/httpd-svn graceful
Try checkout
svn co --username praveen --password <PASSWORD> http://MySubversionIndiaServer/svn/projects/dummyproject/trunk
Edit a file and try commting changes
Eg:
svn commit -m "Testing SVN commit" test.txt
Try browsing
http://MySubversionIndiaServer/svn/
cvs2svn:
Install
cd /home/myname/svn-install-scripts/
wget http://cvs2svn.tigris.org/files/documents/1462/39919/cvs2svn-2.0.1.tar.gz
tar zxvf cvs2svn-2.0.1.tar.gz
cd cvs2svn-2.0.1
make install DESTDIR=/home/myname/svn/softwares/cvs2svn-2.0.1
cd /home/myname/svn/softwares/cvs2svn-2.0.1/usr/bin
ln -s /home/myname/svn/softwares/cvs2svn-2.0.1/usr/lib/python2.3/site-packages/cvs2svn* /usr/lib/python2.3/site-packages/
Providing CVS data locally
Install CVS and copy your repo from elsewhere existing CVS. csv2svn requires CVS & SVN locally.
Conversion
/home/myname/svn/softwares/cvs2svn-2.0.1/usr/bin/cvs2svn
--dry-run
--svnrepos /home/myname/svn/projects/dummyproject_convertedto_svn
/home/myname/cvs/cvsroot/dummyproject_on_cvs
Actual run
/home/myname/svn/softwares/cvs2svn-2.0.1/usr/bin/cvs2svn
--svnrepos
/home/myname/svn/projects/dummyproject_convertedto_svn
/home/myname/cvs/cvsroot/dummyproject_on_cvs
> testrun.log
cvs2svn passes:
/home/myname/svn/softwares/cvs2svn-2.0.1/usr/bin/cvs2svn --help-passes
PASSES:
1 : CollectRevsPass
2 : CollateSymbolsPass
3 : FilterSymbolsPass
4 : SortRevisionSummaryPass
5 : SortSymbolSummaryPass
6 : InitializeChangesetsPass
7 : BreakRevisionChangesetCyclesPass
8 : RevisionTopologicalSortPass
9 : BreakSymbolChangesetCyclesPass
10 : BreakAllChangesetCyclesPass
11 : TopologicalSortPass
12 : CreateRevsPass
13 : SortSymbolsPass
14 : IndexSymbolsPass
15 : OutputPass
Also, you can run multiple passes like:
/home/myname/svn/softwares/cvs2svn-2.0.1/usr/bin/cvs2svn
--verbose
--no-prune
--pass=1
--svnrepos
/home/myname/svn/projects/dummyproject_convertedto_svn
/home/myname/cvs/cvsroot/dummyproject_on_cvs
> testrun.pass1.log
to
/home/myname/svn/softwares/cvs2svn-2.0.1/usr/bin/cvs2svn
--verbose
--no-prune
--pass=15
--svnrepos
/home/myname/svn/projects/dummyproject_convertedto_svn
/home/myname/cvs/cvsroot/dummyproject_on_cvs
> testrun.pass15.log
SELinux, httpd ( apache ), file context ( httpd_sys_content_t ) and other settings
SELinux when enforced can make a lot of things not work. Features developed at
an server(SELinux disabled) may not work when it goes to production. SELinux controls
httpd (Apache) and I can list down few pointers for you.
Errors:
- You see errors when you try creating a VirtualHost having its DocumentRoot on directories other than /var/www/html
- You see errors when you try starting apache on non standard ports other than ones listed below
http_cache_port_t tcp 3128, 8080, 8118
http_cache_port_t udp 3130
http_port_t tcp 80, 443, 488, 8008, 8009, 8443
pegasus_http_port_t tcp 5988
pegasus_https_port_t tcp 5989 - Your script does not execute if you are trying to run system files in it.
Eg:
You are trying to run /usr/bin/crontab, etc from php.
So.. what are we supposed to do? There are two possibilities.
- Disable selinux (refer SELINUX Restrictions On Apache). But kindly dont since it is not required.
- Or set SElinux to allow what you wan't
SELinux can be configured by setting selinux booleans and also by changing the context of the scripts. So.. what is this context? You will come to know below.
If SELINUX is not disabled.. we may have to look into
cat /var/log/messages | grep SELinux
and later accordingly, allow selinux to relax restrictions on httpd.
Manual:
man 8 httpd_selinux
Is SELinux enabled?
dmesg | grep selinux
cat /selinux/enforce
To see processes protected by selinux:
ps -ZC httpd
For entire list
ps -eZ
File attributes
ls -Z /usr/bin/crontab
-rwxr-xr-x root root system_u:object_r:bin_t /usr/bin/crontab Selinux prevents:
ls -Z /home/praveen/info.php
-rw-rw-r-- praveen praveen user_u:object_r:user_home_t /home/praveen/test.php
Selinux allows:
ls -Z /home/praveen/info.php
-rw-rw-r-- praveen praveen user_u:object_r:httpd_sys_content_t /home/praveen/test.php
Types avaliable for apache
getsebool -a | grep httpd
List httpd portsallow_httpd_anon_write --> off
allow_httpd_bugzilla_script_anon_write --> off
allow_httpd_mod_auth_pam --> off
allow_httpd_nagios_script_anon_write --> off
allow_httpd_squid_script_anon_write --> off
allow_httpd_sys_script_anon_write --> off
httpd_builtin_scripting --> on
httpd_can_network_connect --> off
httpd_can_network_connect_db --> off
httpd_can_network_relay --> off
httpd_disable_trans --> off
httpd_enable_cgi --> on
httpd_enable_ftp_server --> off
httpd_enable_homedirs --> on
httpd_rotatelogs_disable_trans --> off
httpd_ssi_exec --> off
httpd_suexec_disable_trans --> off
httpd_tty_comm --> off
httpd_unified --> on
semanage port -l | grep http
http_cache_port_t tcp 3128, 8080, 8118
http_cache_port_t udp 3130
http_port_t tcp 80, 443, 488, 8008, 8009, 8443
pegasus_http_port_t tcp 5988
pegasus_https_port_t tcp 5989
Add a httpd port to selinux
Add it under httpd.conf
Listen 81
Do a graceful.
semanage port -a -t http_port_t -p tcp 81
audit2allow is a perl script that interprets the selinux errors and constructs the right rules to overcome various problems
/usr/bin/audit2allow -i /var/log/messages
sealert -l <id>
(id from /var/log/messages)
Change file context to make it accesible by httpd:
chcon -h root:object_r:httpd_sys_content_t test.php
Sometimes restorecon -v test.php can work.
Recursively,
restorecon -Rv <dir>
The below command will allow apache to access your directory(non default)
chcon -Rt httpd_sys_content_t <your document root dir>
The command below will allow apache to access user home directories as document root:
setsebool -P httpd_enable_homedirs=1
To use system commands thru php or other scripts on apache.. that command's context should be changed to httpd_unconfined_script_exec_t.
Eg:
chcon -t httpd_unconfined_script_exec_t /usr/bin/crontab
For other, undiscussed policy related errors:
tail -f /var/log/messages | grep SELinux
sealert -l <id>
will tell you what to do.
Last option will be to disable SELinux protection to apache by:setsebool -P httpd_disable_trans 1
/etc/init.d/httpd restart
So disabling SELinux is not the solution if you have Apache issues.
PHP big file upload and upload_max_filesize setting
If you want to control the maximum upload file size limit in php don't try doing it
thru script like: ini_set('UPLOAD_MAX_FILESIZE', <value>);
ini_set('POST_MAX_SIZE', <value>);
Instead set the values in php.ini like:
upload_max_filesize = 256M ;(it is 2M by default)
post_max_size = 256M
Do a httpd graceful. It should work.
Reiserfs on CentOS
In CentOS-Base.repo change enabled=1 for centosplus
#additional packages that extend functionality of existing packages
[centosplus]
name=CentOS-$releasever - Plus
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus
#baseurl=http://mirror.centos.org/centos/$releasever/centosplus/$basearch/
gpgcheck=1
enabled=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-centos4
yum -y install reiserfs-utils.x86_64
yum -y install kernel-smp.x86_64 kernel-smp-devel.x86_64
Set default kernel to be loaded as 2.6.9-55.0.6.plus.c4smp
vi /etc/grub.conf
default=0Comment the mount which needs to be chaged from ext3 to reiserfs
timeout=5
splashimage=(hd0,0)/boot/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.9-55.0.6.plus.c4smp)
Edit /etc/fstab
#LABEL=/home /home ext3 defaults 1 2reboot
mkfs.reiserfs /dev/sda3 --label=/home -d
mount /dev/sda3 /home
Edit /etc/fstab
LABEL=/home /home reiserfs defaults 1 2
Thats all!
Common Errors and Resolutions
Common errors, its context and its resolution(fix,dirtyfix):
Error:
rsync: recv_generator: mkdir "" failed: Too many links (31)
rsync: stat "" failed: No such file or directory (2)
rsync: mkstemp ".MXiMwF" failed: No such file or directory (2)
Resolution:
Go for reiserfs or gfs or comply with ext3 subdirectories limitation
Context:
svnsync (reversal)
Error:
svnsync: PROPFIND of '/mysvn': Could not resolve hostname `myhost'
Resolution:
ip instead of hostname
Context:
svn-python
>>>import svn.repos
Error:
undefined symbol: gss_delete_sec_context
Resolution:
Edit Makefile
#SVN_APR_LIBS = path-to-apache-x.x.x/lib/libapr-x.la -luuid -lrt -lcrypt -lpthread -ldl
SVN_APR_LIBS = path-to-apache-x.x.x/lib/libapr-1.la -luuid -lrt -lcrypt -lpthread -ldl -lgssapi
ln -s /usr/lib/libgssapi.so.x.x.x /usr/lib/libgssapi.so
And make again.
Context:
svnsync (reversal)
Error:
svnsync: PROPFIND of '/mysvn': Could not resolve hostname `myhost'
Resolution:
ip instead of hostname
Context:
MIME-tools: /usr/bin/perl -MCPAN -e 'install MIME::Parser'
Error:
No IO::File
Undefined subroutine &Mail::Internet::mailaddress
Resolution:
cpan> install IO::File
cpan> force install MIME::Parser
Context:
Chart: /usr/bin/perl -MCPAN -e 'install Chart::Base'
Error:
The module Chart::Base isn't available on CPAN
Resolution:
wget http://search.cpan.org/CPAN/authors/id/C/CH/CHARTGRP/Chart-2.4.1.tar.gz
tar zxvf Chart-2.4.1.tar.gz
cd Chart-2.4.1
perl Makefile.PL
make
make test
make install
Error:
Starting httpd: Warning: DocumentRoot [path-to-bugzilla] does not exist
Forbidden
You don't have permission to access /README on this server.
Additionally, a 403 Forbidden error was encountered while trying to use an
ErrorDocument to handle the request.
Resolution:
sestatus
cat /selinux/enforce
echo 0 >/selinux/enforce
newrole -r sysadm_r
cat /selinux/enforce
vi /etc/selinux/config
SELINUX=disabled
reboot
Context:
mysqlhotcopy -u <user> -p <pass> <bd> --debug
Error:
Using copy suffix '_
Filtering tables with '(?-xism:.*)'
Invalid db.table name 'db.table`.`field' at
path-to/mysqlhotcopy line 855.
Dirty fix:
Added a new line
vi path-to/mysqlhotcopy
:836
map { s/^.*?.//o } @dbh_tables;
Context:
mysqlhotcopy -u <user> -p <pass> <bd> --debug --addtodest
Error:
DBD::mysql::db do failed: Access denied; you need the RELOAD privilege for
this operation at path-to/mysqlhotcopy line 473.
Resolution:
mysql>
GRANT RELOAD ON *.* TO user@localhost; FLUSH PRIVILEGES;
Context:
Can't connect to the database.
Error:
Too many connections
Is your database installed and up and running?
Do you have the correct username and password selected in localconfig?