Setting up Perforce Replication

This was oddly tricky. There’s not a ton of documentation on the internet about it, at least not that my Google searching turned up. Loads of official Perforce docs, yes, but I kept having problems with those step-by-steps.

Not to fear though. Below is a step-by-step that worked for me. I only tested it on the latest version though, but it seems older versions (after 2012 or so) are the same.

Set up Perforce replication

Master here is at 10.1.5.31
Replica host is at 10.1.5.35
Replication service user is named repl_user_for_Replica3
Assumes your perforce installs are in the same directory (/p4root in this case)
Assumes you installed perforce to run under the “perforce” OS user (default from apt-get)
Assumes both your perforce instances use the same case sensitivity and unicode settings

## Set up the perforce repo and update apt
see: http://capnjosh.com/blog/installing-perforce-on-ubuntu-server-16/

## Do a basic install on the master
apt-get install helix-p4d -y

## On the master, do a basic configuration, setting the p4 root to /p4root
## Don’t use ssl:1666 just type in 1666 for that step
/opt/perforce/sbin/configure-helix-p4d.sh

## On the master, set up a user for the replica to use
p4 -p 10.1.5.31:1666 -u super user -f repl_user_for_Replica3
## Add a line at the bottom with “Type: service”

## On the master, set a password for the service account
p4 -p 10.1.5.31:1666 -u super passwd repl_user_for_Replica3

## On the master, set up a group for the service accounts and make login tickets effectively never expire
p4 -p 10.1.5.31:1666 -u super group service_group
## Users: repl_user_for_Replica3
## Timeout: unlimited
## You can add more users by just having one per line
## You should see “Group service_group created.”

## Add super permissions to the service_group group – sounds alarming, but it seems to be required
## super group service_group * /…
p4 -p 10.1.5.31:1666 -u super protect

## Alright, the idea here is to set all the important replication things on the master rather than on the replica
## This way, for the replica we only have to worry about restoring the checkpoint, copying the versioned files, and naming the replica
## This approach will result in the replica automatically pulling everything it needs after merely starting it up
## I believe you could then start up any other p4 instance and merely point it at the master
## Based on its name it will then have all the replciation configs already set up for it to use

## On the master, set the master for the replica, the p4 instance of which we will call “Replica3”
## also set a lot of other variables the replica will pick up and use
p4 -p 10.1.5.31:1666 -u super configure set Replica3#P4TARGET=10.1.5.31:1666
p4 -p 10.1.5.31:1666 -u super configure set Replica3#P4LOG=replica3Log.txt
p4 -p 10.1.5.31:1666 -u super configure set Replica3#server=3
p4 -p 10.1.5.31:1666 -u super configure set Replica3#monitor=1
p4 -p 10.1.5.31:1666 -u super configure set “Replica3#startup.1=pull -i 1”
p4 -p 10.1.5.31:1666 -u super configure set “Replica3#startup.2=pull -u -i 1”
p4 -p 10.1.5.31:1666 -u super configure set “Replica3#startup.3=pull -u -i 1”
p4 -p 10.1.5.31:1666 -u super configure set Replica3#db.replication=readonly
p4 -p 10.1.5.31:1666 -u super configure set Replica3#lbr.replication=readonly
p4 -p 10.1.5.31:1666 -u super configure set Replica3#serviceUser=repl_user_for_Replica3

## On the master, checkpoint the master
p4 -p 10.1.5.31:1666 -u super admin checkpoint

## Now, copy the checkpoint.3 (or whatever the biggest number is) and then copy that to the replica’s p4root folder
## Yes, it’s likely a 2-step process, where you scp to /tmp, then ssh into the replica host and move from /tmp to /p4root
## Yes, you will have to mkdir /p4root on the replica
scp /p4root/checkpoint.3 ts@10.1.5.35:/tmp

## Copy all the versioned files from the master to the replica’s p4root folder (same location as on the master, relative to p4root)
## Yes, it’s likely a 2-step process, where you scp to /tmp, then ssh into the replica host and move from /tmp to /p4root
## Yes, you will have to mkdir /p4root on the replica
scp -r /p4root/streamsDepot ts@10.1.5.35:/tmp
## I’m using “streamsDepot” because that was the default folder it created. You may have other folders there; each represents a depot

## On the replica install p4
apt-get install helix-p4d -y

## One the replica, create /p4root
mkdir /p4root

## Move or copy the checkpoint we copied earlier
cp /tmp/checkpoint.1 /p4root

## Move or copy the versioned files we copied earlier
cp -r /tmp/streamsDepot /p4root

## On the replica, make sure the perforce user can act on the files you copied over
chown -R perforce:perforce /p4root

## Change users to the perforce OS user to start up the p4d process
su perforce

## On the replica host restore the checkpoint
p4d -r /p4root -jr /p4root/checkpoint.3
## You should see “Recovering from /p4root/checkpoint.3…”
## You should then see a bunch of files that start with “db.” on /p4root
## After you’re all done, btw, you can remove these checkpoint.3 files

## On the replica host log in to the master p4 instance with the service account so you get a forever-ticket, as we had configured on the master
p4 -u repl_user_for_Replica3 -p 10.1.5.31:1666 login

## You should see a .p4tickets file in the perforce user’s home directory when you do “ls -a” after this.
## If not, check the replica3Log.txt file for indicators of what’s wrong.

## Start the replica, naming it Replica3 and having it listen on port 22222
p4d -r /p4root -In Replica3 -p 10.1.5.32:22222 -d
## Naming it “Replica3” means it’ll pull up the configs you set on the master for “Replica3”

## You should connect to the master, submit some changes and confirm you see those in the replica
## In my case, I just copied and pasted and submitted some random text files in the streamsDepot and verified the versioned files were replicated over

## Check to make sure the replication processes are running on the replica
p4 -p 10.1.5.32:22222 -u super monitor show -a

## I *think* you can just have a cron job running that tries to start up p4d as though you were just starting the replica
## (sledgehammer method of making sure it’s always running… hah)
p4d -r /p4root -In Replica3 -p 10.1.5.32:22222 -d

## Potential improvements:
– have .p4tickets file stored in /p4root instead of the perforce user’s home directory (maybe better?)
– See if there’s any way the replication service group could use something less than “super” privileges
– Monitoring to detect if and when replication goes down or otherwise stops
– Cron job to make sure the replica p4d process is auto-restarted if it stops

## Links
## https://www.perforce.com/perforce/doc.current/manuals/p4dist/chapter.replication.html
## http://answers.perforce.com/articles/KB/2520/?q=reseed&l=en_US&fs=Search&pn=1

The replica should then start auto-syncing everyting
In replica3Log.txt, it should be very clean:

Perforce server info:
Perforce Server starting 2017/04/11 01:54:54 pid 12986 P4D/LINUX26X86_64/2016.2/1498579.
Perforce server info:
2017/04/11 01:54:54 pid 12987 repl_user_for_Replica2@unknown background [p4d/2016.2/LINUX26X86_64/1498579] ‘pull -i 1’
Perforce server info:
2017/04/11 01:54:54 pid 12988 repl_user_for_Replica2@unknown background [p4d/2016.2/LINUX26X86_64/1498579] ‘pull -u -i 1’
Perforce server info:
2017/04/11 01:54:54 pid 12989 repl_user_for_Replica2@unknown background [p4d/2016.2/LINUX26X86_64/1498579] ‘pull -u -i 1’

The following message in replica3Log.txt means your repl_user_for_Replica2 doesn’t have super permissions
check out the “protect” step earlier; the replication user group needs to have super permissions on everything

Perforce server error:
2017/04/08 18:01:00 pid 1699 service@10.1.5.32:22222 background ‘pull -i 1’
Startup command failed: client-Message
2017/04/08 18:01:00 pid 1699 service@10.1.5.32:22222 background ‘pull -i 1’
Replica access refused. Ensure that the serverid and service user are correctly configured on the replica. Server Replica1 may not be used by service user service.
Server ‘Replica1’ doesn’t exist.

If you started up p4d on the replica with the wrong IP the replica3Log.txt file will say as much