LilyPond — Contributor’s Guide

Appendices

1. Introduction to contributing

This chapter presents a quick overview of ways that people can help LilyPond.

1.1 Help us

We need you!

Thank you for your interest in helping us — we would love to see you get involved! Your contribution will help a large group of users make beautifully typeset music.

Even working on small tasks can have a big impact: taking care of them allows experienced developers work on advanced tasks, instead of spending time on those simple tasks.

For a multi-faceted project like LilyPond, sometimes it’s tough to know where to begin. In addition to the avenues proposed below, you can send an e-mail to the lilypond-devel@gnu.org mailing list, and we’ll help you to get started.

Simple tasks

No programming skills required!

• Mailing list support: answer questions from fellow users.
• Bug reporting: help users create proper Bug reports, and/or join the Bug Squad to organize Issues.
• Documentation: small changes can be proposed by following the guidelines for Documentation suggestions.
• LilyPond Snippet Repository (LSR): create and fix snippets following the guidelines in Adding and editing snippets.
• Discussions, reviews, and testing: the developers often ask for feedback about new documentation, potential syntax changes, and testing new features. Please contribute to these discussions!

Advanced tasks

These jobs generally require that you have the source code and can compile LilyPond.

Contributors using Linux or FreeBSD may also use Lilydev, but if they prefer their own development environment, they should read Working with source code, and Compiling.

Begin by reading Summary for experienced developers.

• Documentation: for large changes, see Documentation work.
• Website: the website is built from the normal documentation source. See the info about documentation, and also Website work.
• Translations: see Translating the documentation, and Translating the website.
• Bugfixes or new features: read Programming work.

1.2 Overview of work flow

Git is a version control system that tracks the history of a program’s source code. The LilyPond source code is maintained as a Git repository, which contains:

• all of the source files needed to build LilyPond, and
• a record of the entire history of every change made to every file since the program was born.

The ‘official’ LilyPond Git repository is hosted by the GNU Savannah software forge at http://git.sv.gnu.org.

Changes made within one contributor’s copy of the repository can be shared with other contributors using patches. A patch is a text file that indicates what changes have been made. If a contributor’s patch is approved for inclusion (usually through the mailing list), someone on the current development team will push the patch to the official repository.

The Savannah software forge provides two separate interfaces for viewing the LilyPond Git repository online: cgit and gitweb.

Git is a complex and powerful tool, but tends to be confusing at first, particularly for users not familiar with the command line and/or version control systems. We have created the lily-git graphical user interface to ease this difficulty.

Compiling (‘building’) LilyPond allows developers to see how changes to the source code affect the program itself. Compiling is also needed to package the program for specific operating systems or distributions. LilyPond can be compiled from a local Git repository (for developers), or from a downloaded tarball (for packagers). Compiling LilyPond is a rather involved process, and most contributor tasks do not require it.

Contributors can contact the developers through the ‘lilypond-devel’ mailing list. The mailing list archive is located at http://lists.gnu.org/archive/html/lilypond-devel/. If you have a question for the developers, search the archives first to see if the issue has already been discussed. Otherwise, send an email to lilypond-devel@gnu.org. You can subscribe to the developers’ mailing list here: http://lists.gnu.org/mailman/listinfo/lilypond-devel.

1.3 Summary for experienced developers

If you are already familiar with typical open-source tools, here’s what you need to know:

• source repository: hosted by GNU savannah.

  http://git.savannah.gnu.org/gitweb/?p=lilypond.git
  

• environment variables: many maintenance scripts, and many instructions in this guide rely on predefined Environment variables.
• mailing lists: given on Contact.
• branches:
  • master: base your work from this, but do not push to it.
  • staging: after a successful review (see below), push here.
  • translation: translators should base their work from this, and also push to it.
  • dev/foo: feel free to push any new branch name under dev/.
• regression tests: also known as “regtests”; this is a collection of more than a thousand .ly files. We track the output of those files between versions.

  If a patch introduces any unintentional changes to the regtests, we will likely reject it – make sure that you are aware and can explain any regtest changes. More info in Regression tests.

• reviews: after finishing work on a patch or branch:
  1. upload it with our custom git-cl. In addition to uploading it to the google rietveld code review tool, this adds a tracker issue so that we don’t lose your patch. The “status” of your patch is kept on the issue tracker; see Issues.

     https://github.com/gperciva/git-cl
     

     Your patch will be given Patch-new status. More info in Uploading a patch for review.

  2. If your patch passes some automatic tests, it will be given Patch-review status. This generally happens within 24 hours.
  3. After that, the patch must wait for the next “patch countdown”, which occur 3 times a week. If there are a lot of patches waiting for a countdown, a subset of patches are chosen randomly. When your patch is put on a countdown, it will be given Patch-countdown status.
  4. The countdown is a 48-hour period which gives other developers one last chance to review the patch. If no significant problems are found, your patch will be given Patch-push status.
  5. You may now either push it to the staging branch, or email your patch (created with git format-patch) to somebody who will push it for you.

  Advanced note: Yes, this process means that most patches wait between 60-120 hours before reaching master. This is unfortunate, but given our limited resources for reviewing patches and a history of unintended breakage in master, this is the best compromise we have found.

1.4 Mentors

We have a semi-formal system of mentorship, similar to the medieval “journeyman/master” training system. New contributors will have a dedicated mentor to help them “learn the ropes”.

Contributor responsibilities

1. Ask your mentor which sections of the CG you should read.
2. If you get stuck for longer than 10 minutes, ask your mentor. They might not be able to help you with all problems, but we find that new contributors often get stuck with something that could be solved/explained with 2 or 3 sentences from a mentor.
3. If you have been working on a task much longer than was originally estimated, stop and ask your mentor. There may have been a miscommunication, or there may be some time-saving tips that could vastly simply your task.
4. Send patches to your mentor for initial comments.
5. Inform your mentor if you’re going to be away for a month, or if you leave entirely. Contributing to lilypond isn’t for everybody; just let your mentor know so that we can reassign that work to somebody else.
6. Inform your mentor if you’re willing to do more work – we always have way more work than we have helpers available. We try to avoid overwhelming new contributors, so you’ll be given less work than we think you can handle.

Mentor responsibilities

1. Respond to questions from your contributor(s) promptly, even if the response is just “sorry, I don’t know” or “sorry, I’m very busy for the next 3 days; I’ll get back to you then”. Make sure they feel valued.
2. Inform your contributor(s) about the expected turnaround for your emails – do you work on lilypond every day, or every weekend, or what? Also, if you’ll be unavailable for longer than usual (say, if you normally reply within 24 hours, but you’ll be at a conference for a week), let your contributors know. Again, make sure they feel valued, and that your silence (if they ask a question during that period) isn’t their fault.
3. Inform your contributor(s) if they need to do anything unusual for the builds, such as doing a “make clean / doc-clean” or switching git branches (not expected, but just in case...)
4. You don’t need to be able to completely approve patches. Make sure the patch meets whatever you know of the guidelines (for doc style, code indentation, whatever), and then send it on to -devel for more comments. If you feel confident about the patch, you can push it directly (this is mainly intended for docs and translations; code patches should almost always go to -devel before being pushed).
5. Keep track of patches from your contributor. Either upload them to Rietveld yourself, or help+encourage them to upload the patches themselves. When a patch is on Rietveld, it’s your responbility to get comments for it, and to add a link to the patch to the google tracker. (tag it “patch-new”, or “patch-review” if you feel very confident in it)
6. Encourage your contributor to review patches, particularly your own! It doesn’t matter if they’re not familiar with C++ / scheme / build system / doc stuff – simply going through the process is valuable. Besides, anybody can find a typo!
7. Contact your contributor at least once a week. The goal is just to get a conversation started – there’s nothing wrong with simply copy&pasting this into an email:

   Hey there,
   
   How are things going?  If you sent a patch and got a review, do
   you know what you need to fix?  If you sent a patch but have no
   reviews yet, do you know when you will get reviews?  If you are
   working on a patch, what step(s) are you working on?
   

2. Quick start

Want to submit a patch for LilyPond? Great! Never created a patch before? Never compiled software before? No problem! This chapter is for you and will help you do this as quickly and easily as possible.

2.1 LilyDev

There is a disk image of a ‘remix’ of Ubuntu GNU/Linux available for download which includes all the necessary software and tools to compile both LilyPond and the documentation. Called the “Ubuntu LilyPond Developer Remix”, but known simply as “LilyDev” for short. Although it is not possible to compile LilyPond on Windows and extremely difficult on MacOS, LilyDev can be installed and run inside a ‘virtual machine’ on any of these operating systems without disturbing your main operating system. The LilyDev disk image can also be burnt to a DVD and installed like any other Ubuntu GNU/Linux distribution.

Most virtualization software can be used but we recommend VirtualBox as it is available for all major operating systems and is easy to install & configure.

If you are not familiar with GNU/Linux, it may be beneficial to read a couple of “introduction to Ubuntu” web pages.

For those interested, the LilyDev remix is currently based on a 32bit version of 10.04 LTS Ubuntu (Lucid Lynx).

Where to get LilyDev

Download the Ubuntu LilyPond Developer Remix CD image file (approximately 1 GB) from here:

http://www.philholmes.net/lilypond/LilyDev/ubuntu-LilyDev-remix-2.6.iso

Some advanced users might want this file too:

http://www.philholmes.net/lilypond/LilyDev/ubuntu-LilyDev-remix-2.6.iso.md5

(If you don’t recognize what this file is, then you don’t need it.)

An alternate site for obtaining these files is available:

http://www.et.byu.edu/~sorensen/ubuntu-LilyDev-remix-2.6.iso
http://www.et.byu.edu/~sorensen/ubuntu-LilyDev-remix-2.6.iso.md5

Installing LilyDev in VirtualBox

This section discusses how to install and use LilyDev with VirtualBox.

1. Download Virtualbox from here:

   http://www.virtualbox.org/wiki/Downloads
   

   Note: In virtualization terminology, the operating system where Virtualbox is installed is known as the host. LilyDev will be installed ‘inside’ Virtualbox as a guest.

2. Start the VirtualBox software and click ‘New’ to create a new “virtual machine”.

   The ‘New Virtual Machine Wizard’ will walk you through setting up your guest virtual machine. Choose an appropriate name for your LilyDev installation and select the ‘Linux’ operating system. When selecting the ‘version’ use ‘Ubuntu’ if available (but not the ‘64 bit’ option). If you do not have that specific option choose ‘Linux 2.6’ (again do not choose any option that has 64 bit next to it).

3. Select the amount of RAM you will allow the LilyDev guest to use from your host operating system when it is running. If possible, use at least 700 MB of RAM; the more RAM you can spare from your host the better, although LilyDev will currently use no more than 4 GB (4096 MB) even if you are able to assign more.
4. For your ‘Virtual Hard Disk’, leave the ‘Create new hard disk’ option checked, use the default ‘VDI’ and “Dynamically allocated” options for the virtual hard drive. A complete compile of everything (code, docs, regression tests) can reach 10 GB so size your virtual disk and its location accordingly.
5. Verify the summary details and click ‘Create’, when you are satisfied. Your new guest will be displayed in the Virtualbox window. Click the ‘Start’ button and the ‘First Run Wizard’ will prompt you for the installation media. Click the browse icon and locate the LilyDev disk image and click through the wizard to start the installation process.
6. When the LilyDev disk image boots, it shows a prompt:

   ISOLINUX boot:
   

   Hit the Return key (or wait 30 seconds) and then when the installer screen loads, using the arrow keys select ‘Install - start the installer directly’ to begin the install process of LilyDev on your virtual hard disk. The Ubuntu software will walk you through the complete installation process.

7. At the “Prepare disk space” stage, do not be afraid to select “Erase and use the entire disk”, since this refers to your virtual disk, not your machine’s actual hard disk.
8. Click through the rest of the wizard, filling in any appropriate details when asked and wait for the install to complete.

   Note: This will take anywhere from 10 minutes to up to an hour depending on the speed of your computer and if Ubuntu detects you are connected to the internet and needs to download any additional security updates or patches, although these updates are not required to compile LilyPond and it is possible to skip the additional downloads to speed up the install process.

9. When prompted by the Ubuntu installer wizard, restart the virtual machine and then when prompted to ‘eject the CD’ by virtual box, just click inside the virtual machine window and hit the return key to reboot the virtual machine. It will not try to restart the installer but start the virtual machine proper. LilyDev is now installed and running!
10. The current version of LilyPond requires the texlive-lang-cyrillic package. This package is not part of LilyDev 2.6. Add the package to LilyDev with:

    sudo apt-get install texlive-lang-cyrillic
    

Known issues and warnings

Not all hardware is supported in all virtualization tools. In particular, some contributors have reported problems with USB network adapters. If you have problems with network connection (for example Internet connection in the host system is lost when you launch virtual system), try installing and running LilyDev with your computer’s built-in network adapter used to connect to the network. Refer to the help documentation that comes with your virtualization software.

Configuring LilyDev in VirtualBox

VirtualBox has extra ‘guest additions’ which although are not necessary to use LilyDev or compile Lilypond, do provide some additional features to your Virtual Machine to make it easier to work with. Such as being able to dynamically resize the LilyDev window, allow seamless interaction with your mouse pointer on both the host and guest and let you copy/paste between your host and guest if needed.

1. Select the ‘Devices’ menu from the virtual machine window and choose ‘Install Guest Additions...’. This will automount a CD which will prompt you to autorun it. Click OK and follow the instructions. It is recommended to reboot the guest when the installation is complete.

   Other virtualization software will also have their own ‘guest’ additions, follow the normal procedures for your virtualization software with Ubuntu as the client.

2. Restart Ubuntu to complete the installation of the guest additions.

   Advanced note: If you do any kernel upgrades, you may need to reinstall the additional software. Just follow the step above again and reboot when the reinstallation is complete.

Other items that may be helpful:

• In the settings for the virtual machine, set the network to Bridged mode to allow you to access shared folders when using Windows hosts.
• Set up any additional features, such as ‘Shared Folders’ between your main operating system and Ubuntu. This is distinct from the networked share folders in Windows. Consult the external documentation for this.

  Some longtime contributors have reported that ‘shared folders’ are rarely useful and not worth the fuss, particularly since files can be shared over a network instead.

• Pasting into a terminal is done with Ctrl+Shift+v.
• The “Places” top-menu has shortcuts to a graphical “navigator” like Windows Explorer or the MacOS X Finder.
• Right-click allows you to edit a file with gedit. We recommend using gedit.
• One particular change from Windows and MacOS X is that most software should be installed with your “package manager”; this vastly simplifies the process of installing and configuring software. Go to Applications → Ubuntu Software Center.

2.2 lily-git

The ‘LilyPond Contributor’s Git Interface’ (otherwise known as lily-git.tcl) is a simple-to-use GUI to help you download and update the LilyPond source code as well as an aid to making software patches.

Where to get lily-git

Depending on your development environment, lily-git may already be installed on your computer.

• If you are using LilyDev (see LilyDev) then lily-git is already installed and ready to run.
• For those not using LilyDev then lily-git can be obtained by downloading the software directly. See Manually installing lily-git.tcl.
• Finally, lily-git is always part of the LilyPond source code and is located in ‘$LILYPOND_GIT/scripts/auxillar/lily-git.tcl’.

Configuring lily-git and downloading the source code

1. Type (or copy&paste) into the Terminal:

   lily-git.tcl
   

   You will be prompted to enter your name and your email address. This information is used only to identify and label any patches you create. This information can be edited if required later. See Configuring Git. Click on the Submit button to update lily-git with this information.

2. Click on the “Get source” button.

   A directory called ‘$LILYPOND_GIT’ is now created within your home directory and the complete source code will start to be downloaded into it.

   Note: Be patient! The complete source is around 150 Mb.

   When the source code has been downloaded, the “Command output” window in lily-git will update and display “Done” on the very last line. The button label will change to say “Update source”.

   Note: Some contributors have reported that occasionally nothing happens at this step at all. If this occurs, then try again in a few minutes – it could be an intermittant network problem. If the problem persists, please ask for help.

3. Close the lily-git GUI and navigate to the ‘$LILYPOND_GIT’ directory to view and edit the source files.

If this is the first time you have compiled LilyPond then please go to Compiling with LilyDev before reading on.

How to use lily-git

1. Update source

At the beginning of each session of lilypond work, you should click the “Update source” button to get the latest changes to the source code.

2a. New local commit

A single commit typically represents one logical set of related changes (such as a bug-fix), and may incorporate changes to multiple files at the same time.

When you’re finished making the changes for a commit, click the “New local commit” button. This will open the “Git Commit Message” window. The message header is required, and the message body is optional.

After entering a commit message, click “OK” to finalize the commit.

2b. Amend previous commit

You can go back and make changes to the most recent commit with the “Amend previous commit” button. This is useful if a mistake is found after you have clicked the “New local commit” button.

To amend the most recent commit, re-edit the source files as needed and then click the “Amend previous commit” button. The earlier version of the commit is not saved, but is replaced by the new one.

3. Make patch set

Before making a patch set from any commits, you should click the “Update source” button to make sure the commits are based on the most recent remote snapshot.

When you click the “Make patch set” button, lily-git.tcl will produce patch files for any new commits, saving them to the current directory. The command output will display the name of the new patch files near the end of the output:

0001-CG-add-lily-git-instructions.patch
Done.

Send patch files to the appropriate place:

• If you have a mentor, send it to them via email.
• Translators should send patches to translations@lilynet.net.
• More experienced contributors should upload the patch for web-based review. This requires additional software and use of the command-line; see Uploading a patch for review.
• If you have trouble uploading the patch for review, ask for help on lilypond-devel@gnu.org.

The “Abort changes – Reset to origin” button

The button labeled “Abort changes – Reset to origin” will copy all changed files to a subdirectory of ‘$LILYPOND_GIT’ named ‘aborted_edits/’, and will reset the repository to the current state of the remote repository (at git.sv.gnu.org).

2.3 Compiling with LilyDev

LilyDev is our ‘remix’ of Ubuntu which contains all the necessary dependencies to do lilypond development; for more information, see LilyDev.

Preparing the build

To prepare the build directory, enter (or copy&paste) the below text. This should take less than a minute.

cd $LILYPOND_GIT
sh autogen.sh --noconfigure
mkdir -p build/
cd build/
../configure

Building lilypond

Compiling lilypond will likely take between 5 and 60 minutes, depending on your computer’s speed and available RAM. We recommend that you minimize the terminal window while it is building; this can have a non-negligible effect on compilation speed.

cd $LILYPOND_GIT/build/
make

You may run the compiled lilypond with:

cd $LILYPOND_GIT/build/
out/bin/lilypond my-file.ly

Building the documentation

Compiling the documentation is a much more involved process, and will likely take 2 to 10 hours.

cd $LILYPOND_GIT/build/
make
make doc

The documentation is put in ‘out-www/offline-root/’. You may view the html files by entering the below text; we recommend that you bookmark the resulting page:

firefox $LILYPOND_GIT/build/out-www/offline-root/index.html

Installing

Don’t. There is no reason to install lilypond within LilyDev. All development work can (and should) stay within the ‘$LILYPOND_GIT’ directory, and any personal composition or typesetting work should be done with an official GUB release.

Problems and other options

To select different build options, or isolate certain parts of the build, or to use multiple CPUs while building, read Compiling.

In particular, contributors working on the documentation should be aware of some bugs in the build system, and should read the workarounds in Generating documentation.

2.4 Now start work!

LilyDev users may now skip to the chapter which is aimed at their intended contributions:

• Documentation work
• Translating the documentation
• Website work
• Regression tests
• Programming work

These chapters are mainly intended for people not using LilyDev, but they contain extra information about the “behind-the-scenes” activities. We recommend that you read these at your leisure, a few weeks after beginning work with LilyDev.

• Working with source code
• Compiling

3. Working with source code

Advanced contributors will find this material quite useful, particularly if they are working on major new features.

3.1 Manually installing lily-git.tcl

We have created an easy-to-use GUI to simplify git for new contributors. If you are comfortable with the command-line, then skip ahead to Starting with Git.

1. If you haven’t already, download and install Git.
   • Windows users: download the .exe file labeled “Full installer for official Git” from:

     http://code.google.com/p/msysgit/downloads/list
     

   • Other operating systems: either install git with your package manager, or download it from the “Binaries” section of:

     http://git-scm.com/download
     

2. Download the lily-git.tcl script from:

   http://git.sv.gnu.org/cgit/lilypond.git/plain/scripts/auxiliar/lily-git.tcl

3. To run the program from the command line, navigate to the directory containing lily-git.tcl and enter:

   wish lily-git.tcl
   

4. Click on the “Get source” button.

   This will create a directory called ‘lilypond-git/’ within your home directory, and will download the source code into that directory (around 150 Mb). When the process is finished, the “Command output” window will display “Done”, and the button label will change to say “Update source”.

5. Navigate to the ‘lilypond-git/’ directory to view the source files.

Further instructions are in How to use lily-git.

3.2 Starting with Git

Using the Git program directly (as opposed to using the lily-git.tcl GUI) allows you to have much greater control over the contributing process. You should consider using Git if you want to work on complex projects, or if you want to work on multiple projects concurrently.

3.2.1 Setting up

Installing Git

If you are using a Unix-based machine, the easiest way to download and install Git is through a package manager such as rpm or apt-get – the installation is generally automatic. The only required package is (usually) called git-core, although some of the auxiliary git* packages are also useful (such as gitk).

Alternatively, you can visit the Git website (http://git-scm.com/) for downloadable binaries and tarballs.

Initializing a repository

Once Git is installed, get a copy of the source code:

git clone git://git.sv.gnu.org/lilypond.git ~/lilypond-git

The above command will put the it in ‘~/lilypond-git’, where ~ represents your home directory.

Technical details

This creates (within the ‘$LILYPOND_GIT’ directory) a subdirectory called ‘.git/’, which Git uses to keep track of changes to the repository, among other things. Normally you don’t need to access it, but it’s good to know it’s there.

Configuring Git

Before working with the copy of the main LilyPond repository, you should configure some basic settings with the git config command. Git allows you to set both global and repository-specific options.

To configure settings that affect all repositories, use the ‘--global’ command line option. For example, the first two options that you should always set are your name and email, since Git needs these to keep track of commit authors:

git config --global user.name "John Smith"
git config --global user.email john@example.com

To configure Git to use colored output where possible, use:

git config --global color.ui auto

The text editor that opens when using git commit can also be changed. If none of your editor-related environment variables are set ($GIT_EDITOR, $VISUAL, or $EDITOR), the default editor is usually vi or vim. If you’re not familiar with either of these, you should probably change the default to an editor that you know how to use. For example, to change the default editor to nano, enter:

git config --global core.editor nano

Finally, and in some ways most importantly, let’s make sure that we know what branch we’re on. If you’re not using LilyDev, add this to your ‘~/.bashrc’:

export PS1="\u@\h \w\$(__git_ps1)$ "

If you are not using LilyDev, you may need to install the additional git-completion package, but it is definitely worth it.

Technical details

Git stores the information entered with git config --global in the file ‘.gitconfig’, located in your home directory. This file can also be modified directly, without using git config. The ‘.gitconfig’ file generated by the above commands would look like this:

[user]
        name = John Smith
        email = john@example.com
[color]
        ui = auto
[core]
        editor = nano

Using the git config command without the ‘--global’ option configures repository-specific settings, which are stored in the file ‘.git/config’. This file is created when a repository is initialized (using git init), and by default contains these lines:

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true

However, since different repository-specific options are recommended for different development tasks, it is best to avoid setting any now. Specific recommendations will be mentioned later in this manual.

3.2.2 Git for the impatient

Ok, so you’ve been using lily-git.tcl for a while, but it’s time to take the next step. Since our review process delays patches by 60-120 hours, and you want to be able to work on other stuff while your previous work is getting reviewed, you’re going to use branches.

You can think of a branch as being a separate copy of the source code. But don’t worry about it.

Start work: make a new branch

Let’s pretend you want to add a section to the Contributor’s Guide about using branches.

Start by updating the repository, then making a new branch. Call the branch anything you want as long as the name starts with dev/. Branch names that don’t begin with dev/ are reserved for special things in lilypond.

git checkout master
git pull -r origin master
git branch dev/cg

Switch to that branch

Nothing has happened to the files yet. Let’s change into the new branch. You can think of this as “loading a file”, although in this case it’s really “loading a directory and subdirectories full of files”.

git checkout dev/cg

Your prompt now shows you that you’re on the other branch:

gperciva@LilyDev:~/lilypond-git (dev/cg)$

To be able to manage multiple lilypond issues at once, you’ll need to switch branches. You should have each lilypond issue on a separate branch. Switching branches is easy:

git checkout master
git checkout origin/staging
git checkout origin/release/unstable
git checkout dev/cg

Branches that begin with origin/ are part of the remote repository, rather than your local repository, so when you check them out you get a temporary local branch. You should never make changes directly on a branch beginning with origin/. You get changes into the remote repository by making them in local branches, and then pushing them to origin/staging as described below.

Make your changes

Edit files, then commit them.

git commit -a

Remember how I said that switching to a branch was like “loading a directory”? Well, you’ve just “saved a directory”, so that you can “load” it later.

When you create a new file, you need to add it to git, then commit it:

git add input/regression/avoid-crash-on-condition.ly
git commit -a

Edit more files. Commit them again. Edit yet more files, commit them again. Go eat dinner. Switch to master so you can play with the latest changes from other developers. Switch back to your branch and edit some more. Commit those changes.

At this stage, don’t worry about how many commits you have.

Save commits to external files

Branches are nerve-wracking until you get used to them. You can save your hard work as individual ‘.patch’ files. Be sure to commit your changes first.

git commit -a
git format-patch master

I personally have between 4 and 20 of those files saved in a special folder at any point in time. Git experts might laugh as that behavior, but I feel a lot better knowing that I’ve got those backups.

Prepare your branch for review

After committing, you can update your branch with the latest master:

git commit -a
git checkout master
git pull -r origin master
git checkout dev/cg
git rebase master

Due to the speed of lilypond development, sometimes master has changed so much that your branch can no longer be applied to it. In that happens, you will have a merge conflict. Stop for a moment to either cry or have a stiff drink, then proceed to Merge conflicts.

Upload your branch

Finally, you’re finished your changes. Time to upload for review. Make sure that you’re on your branch, then upload:

git checkout dev/cg
git-cl upload master

Wait for reviews

While you’re waiting for a countdown and reviews, go back to master, make a dev/doc-beams branch, and start adding doc suggestions from issue 12345 from the tracker. Or make a dev/page-breaks and fix bug in page breaking. Or whatever. Don’t worry, your dev/cg is safe.

Combining commits (optional unless you have broken commits)

Does the history of your branch look good?

gitk

If you have a lot of commits on your branch, you might want to combine some of them. Alternately, you may like your commits, but want to edit the commit messages.

git rebase -i master

Follow instructions on the screen.

Push to staging

When you’ve got the coveted Patch-push status, time to prepare your upload:

git fetch
git rebase origin/staging dev/cg~0
gitk HEAD 

If everything looks good, push it:

git push origin HEAD:staging

Then change back to your working branch:

git checkout dev/cg

Delete your branch (safe)

After a few hours, if there’s nothing wrong with your branch, it should be automatically moved to origin/master. Update, then try removing your branch:

git checkout master
git pull -r origin master
git branch -d dev/cg

The last command will fail if the contents of dev/cg are not present in origin/master.

Delete your branch (UNSAFE)

Sometimes everything goes wrong. If you want to remove a branch even though it will cause your work to be lost (that is, if the contents of dev/cg are not present in master), follow the instructions in “Delete your branch (safe)”, but replace the -d on the final line with a -D.

3.2.3 Other repositories

We have a few other code repositories.

lilypond-extra

There is a separate repository for general administrative scripts, as well as pictures and media files for the website. People interested in working on the website should download this repository, and set their $LILYPOND_WEB_MEDIA_GIT environment variable to point to that repository.

https://github.com/gperciva/lilypond-extra

To configure an environment variable in bash (the default for most GNU/Linux distributions),

export LILYPOND_WEB_MEDIA_GIT=$HOME/dir/of/lilypond-extra/

Be aware that lilypond-extra is the definitive source for some binary files - in particular PDF versions of papers concerning LilyPond. To add further PDFs of this sort, all that is necessary is to add the PDF to lilypond-extra and then add a reference to it in the documentation. The file will then be copied to the website when make website is run.

However, pictures that are also used in the documentation build are mastered in the main git repository. If any of these is changed, it should be updated in git, and then the updates copied to lilypond-extra.

Grand Unified Builder (GUB)

Another item of interest might be the Grand Unified Builder, our cross-platform building tool. Since it is used by other projects as well, it is not stored in our gub repository. For more info, see http://lilypond.org/gub.

There are two locations for this repository: the version being used to build lilypond, which is at

http://github.com/gperciva/gub

and the original version by Jan Nieuwenhuizen, kept at

http://github.com/janneke/gub

lilypad

Our binary releases on MacOS X and Windows contain a lightweight text editor. This code is here:

https://github.com/gperciva/lilypad

yet more repositories

There are a few other repositories floating around, which will hopefully be documented in the near future.

3.2.4 Downloading remote branches

Organization of remote branches

The main LilyPond repository is organized into branches to facilitate development. These are often called remote branches to distinguish them from local branches you might create yourself (see Using local branches).

The master branch contains all the source files used to build LilyPond, which includes the program itself (both stable and development releases), the documentation (and its translations), and the website. Generally, the master branch is expected to compile successfully.

The translation branch is a side branch that allows translators to work without needing to worry about compilation problems. Periodically, the Translation Meister (after verifying that it doesn’t break compilation), will merge this branch into staging to incorporate recent translations. Similarly, the master branch is usually merged into the translation branch after significant changes to the English documentation. See Translating the documentation for details.

LilyPond repository sources

The recommended source for downloading a copy of the main repository is:

git://git.sv.gnu.org/lilypond.git

However, if your internet router filters out connections using the GIT protocol, or if you experience difficulty connecting via GIT, you can try these other sources:

ssh://git.sv.gnu.org/srv/git/lilypond.git
http://git.sv.gnu.org/r/lilypond.git

The SSH protocol can only be used if your system is properly set up to use it. Also, the HTTP protocol is slowest, so it should only be used as a last resort.

Downloading individual branches

Once you have initialized an empty Git repository on your system (see Initializing a repository), you can download a remote branch into it. Make sure you know which branch you want to start with.

To download the master branch, enter the following:

git remote add -ft master -m master \
  origin git://git.sv.gnu.org/lilypond.git/

To download the translation branch, enter:

git remote add -ft translation -m \
  translation origin git://git.sv.gnu.org/lilypond.git/

The git remote add process could take up to ten minutes, depending on the speed of your connection. The output will be something like this:

Updating origin
remote: Counting objects: 235967, done.
remote: Compressing objects: 100% (42721/42721), done.
remote: Total 235967 (delta 195098), reused 233311 (delta 192772)
Receiving objects: 100% (235967/235967), 68.37 MiB | 479 KiB/s, done.
Resolving deltas: 100% (195098/195098), done.
From git://git.sv.gnu.org/lilypond
 * [new branch]      master     -> origin/master
From git://git.sv.gnu.org/lilypond
 * [new tag]         flower/1.0.1 -> flower/1.0.1
 * [new tag]         flower/1.0.10 -> flower/1.0.10
⋮
 * [new tag]         release/2.9.6 -> release/2.9.6
 * [new tag]         release/2.9.7 -> release/2.9.7

When git remote add is finished, the remote branch should be downloaded into your repository—though not yet in a form that you can use. In order to browse the source code files, you need to create and checkout your own local branch. In this case, however, it is easier to have Git create the branch automatically by using the checkout command on a non-existent branch. Enter the following:

git checkout -b branch origin/branch

where branch is the name of your tracking branch, either master or translation.

Git will issue some warnings; this is normal:

warning: You appear to be on a branch yet to be born.
warning: Forcing checkout of origin/master.
Branch master set up to track remote branch master from origin.
Already on 'master'

By now the source files should be accessible—you should be able to edit any files in the ‘$LILYPOND_GIT’ directory using a text editor of your choice. But don’t start just yet! Before editing any source files, learn how to keep your changes organized and prevent problems later—read Basic Git procedures.

Technical Details

The git remote add command should add some lines to your local repository’s ‘.git/config’ file:

[remote "origin"]
        url = git://git.sv.gnu.org/lilypond.git/
        fetch = +refs/heads/master:refs/remotes/origin/master

Downloading all remote branches

To download all remote branches at once, you can clone the entire repository:

git clone git://git.sv.gnu.org/lilypond.git

Other branches

Most contributors will never need to touch the other branches. If you wish to do so, you will need more familiarity with Git; please see Other Git documentation.

• dev/XYZ: These branches are for individual developers. They store code which is not yet stable enough to be added to the master branch.
• stable/XYZ: The branches are kept for archival reasons.
• archive/XYZ: The branches are kept for archival reasons.

3.3 Basic Git procedures

3.3.1 The Git contributor’s cycle

Here is a simplified view of the contribution process on Git:

1. Update your local repository by pulling the most recent updates from the remote repository.
2. Edit source files within your local repository’s working directory.
3. Commit the changes you’ve made to a local branch.
4. Generate a patch to share your changes with the developers.

3.3.2 Pulling and rebasing

When developers push new patches to the git.sv.gnu.org repository, your local repository is not automatically updated. It is important to keep your repository up-to-date by periodically pulling the most recent commits from the remote branch. Developers expect patches to be as current as possible, since outdated patches require extra work before they can be used.

Occasionally you may need to rework some of your own modifications to match changes made to the remote branch (see Resolving conflicts), and it’s considerably easier to rework things incrementally. If you don’t update your repository along the way, you may have to spend a lot of time resolving branch conflicts and reconfiguring much of the work you’ve already done.

Fortunately, Git is able to resolve certain types of branch conflicts automatically with a process called rebasing. When rebasing, Git tries to modify your old commits so they appear as new commits (based on the latest updates). For a more involved explanation, see the git-rebase man page.

To pull without rebasing (recommended for translators), use the following command:

git pull    # recommended for translators

If you’re tracking the remote master branch, you should add the ‘-r’ option (short for ‘--rebase’) to keep commits on your local branch current:

git pull -r # use with caution when translating

If you don’t edit translated documentation and don’t want to type ‘-r’ every time, configure the master branch to rebase by default with this command:

git config branch.master.rebase true

If pull fails because of a message like

error: Your local changes to 'Documentation/learning/tutorial.itely'
would be overwritten by merge.  Aborting.

or

Documentation/learning/tutorial.itely: needs update
refusing to pull with rebase: your working tree is not up-to-date

it means that you have modified some files in you working tree without committing changes (see Commits and patches); you can use the git stash command to work around this:

git stash      # save uncommitted changes
git pull -r    # pull using rebase (translators omit "-r")
git stash pop  # reapply previously saved changes

Note that git stash pop will try to apply a patch, and this may create a conflict. If this happens, see Resolving conflicts.

TODO: I think the next paragraph is confusing. Perhaps prepare the reader for new terms ‘committish’ and ‘head’? -mp

TODO: when committishes automatic conditional update have been tested and documented, append the following to the warning above: Note that using update-committishes make target generally touches committishes.

Technical details

The git config command mentioned above adds the line rebase = true to the master branch in your local repository’s ‘.git/config’ file:

[branch "master"]
        remote = origin
        merge = refs/heads/master
        rebase = true

3.3.3 Using local branches

Creating and removing branches

Local branches are useful when you’re working on several different projects concurrently. To create a new branch, enter:

git branch name

To delete a branch, enter:

git branch -d name

Git will ask you for confirmation if it sees that data would be lost by deleting the branch. Use ‘-D’ instead of ‘-d’ to bypass this. Note that you cannot delete a branch if it is currently checked out.

Listing branches and remotes

You can get the exact path or URL of all remote branches by running:

git remote -v

To list Git branches on your local repositories, run

git branch     # list local branches only
git branch -r  # list remote branches
git branch -a  # list all branches

Checking out branches

To know the currently checked out branch, i.e. the branch whose source files are present in your working tree, read the first line of the output of

git status

The currently checked out branch is also marked with an asterisk in the output of git branch.

You can check out another branch other_branch, i.e. check out other_branch to the working tree, by running

git checkout other_branch

Note that it is possible to check out another branch while having uncommitted changes, but it is not recommended unless you know what you are doing; it is recommended to run git status to check this kind of issue before checking out another branch.

Merging branches

To merge branch foo into branch bar, i.e. to “add” all changes made in branch foo to branch bar, run

git checkout bar
git merge foo

If any conflict happens, see Resolving conflicts.

There are common usage cases for merging: as a translator, you will often want the Translations meister to merge master into translation; on the other hand, the Translations meister wants to merge translation into staging whenever he has checked that translation builds successfully.

3.3.4 Commits and patches

Understanding commits

Technically, a commit is a single point in the history of a branch, but most developers use the term to mean a commit object, which stores information about a particular revision. A single commit can record changes to multiple source files, and typically represents one logical set of related changes (such as a bug-fix). You can list the ten most recent commits in your current branch with this command:

git log -10 --oneline

If you’re using an older version of Git and get an ‘unrecognized argument’ error, use this instead:

git log -10 --pretty=oneline --abbrev-commit

More interactive lists of the commits on the remote master branch are available at http://git.sv.gnu.org/gitweb/?p=lilypond.git;a=shortlog and http://git.sv.gnu.org/cgit/lilypond.git/log/.

Making commits

Once you have modified some source files in your working directory, you can make a commit with the following procedure:

1. Make sure you’ve configured Git properly (see Configuring Git). Check that your changes meet the requirements described in Code style and/or Documentation policy. For advanced edits, you may also want to verify that the changes don’t break the compilation process.
2. Run the following command:

   git status
   

   to make sure you’re on the right branch, and to see which files have been modified, added or removed, etc. You may need to tell Git about any files you’ve added by running one of these:

   git add file  # add untracked file individually
   git add .     # add all untracked files in current directory
   

   After git add, run git status again to make sure you got everything. You may also need to modify ‘GNUmakefile’.

3. Preview the changes about to be committed (to make sure everything looks right) with:

   git diff HEAD
   

   The HEAD argument refers to the most recent commit on the currently checked-out branch.

4. Generate the commit with:

   git commit -a
   

   The ‘-a’ is short for ‘--all’ which includes modified and deleted files, but only those newly created files that have previously been added.

Commit messages

When you run the git commit -a command, Git automatically opens the default text editor so you can enter a commit message. If you find yourself in a foreign editing environment, you’re probably in vi or vim. If you want to switch to an editor you’re more familiar with, quit by typing :q! and pressing <Enter>. See Configuring Git for instructions on changing the default editor.

In any case, Git will open a text file for your commit message that looks like this:

# Please enter the commit message for your changes.  Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#	modified:   working.itexi
#

Your commit message should begin with a one-line summary describing the change (no more than 50 characters long), and if necessary a blank line followed by several lines giving the details:

Doc: add Baerenreiter and Henle solo cello suites

Added comparison of solo cello suite engravings to new essay with
high-res images, fixed cropping on Finale example.

Commit messages often start with a short prefix describing the general location of the changes. If a commit affects the documentation in English (or in several languages simultaneously) the commit message should be prefixed with “Doc: ”. If the commit affects only one of the translations, the commit message should be prefixed with “Doc-**: ”, where ** is the two-letter language code. Commits that affect the website should use “Web: ” for English, and “Web-**: ” for the other languages. Also, changes to a single file are often prefixed with the name of the file involved. Visit the links listed in Understanding commits for examples.

Making patches

If you want to share your changes with other contributors and developers, you need to generate patches from your commits. We prefer it if you follow the instructions in Uploading a patch for review. However, we present an alternate method here.

You should always run git pull -r (translators should leave off the ‘-r’) before doing this to ensure that your patches are as current as possible.

Once you have made one or more commits in your local repository, and pulled the most recent commits from the remote branch, you can generate patches from your local commits with the command:

git format-patch origin

The origin argument refers to the remote tracking branch at git.sv.gnu.org. This command generates a separate patch for each commit that’s in the current branch but not in the remote branch. Patches are placed in the current working directory and will have names that look something like this:

0001-Doc-Fix-typos.patch
0002-Web-Remove-dead-links.patch
⋮

Send an email (must be less than 64 KB) to lilypond-devel@gnu.org briefly explaining your work, with the patch files attached. Translators should send patches to translations@lilynet.net. After your patches are reviewed, the developers may push one or more of them to the main repository or discuss them with you.

Uploading a patch for review

Any non-trivial change should be uploaded to our “Rietveld” code review website:

http://codereview.appspot.com/

git-cl install

LilyDev users should skip over these ‘install’ instructions.

1. Install git-cl by entering:

   git clone https://github.com/gperciva/git-cl.git
   

   If that command fails for some reason, try this instead:

   git clone git://github.com/gperciva/git-cl.git
   

2. Add the ‘git-cl/’ directory to your PATH, or create a symbolic link to the git-cl and upload.py scripts in one of your PATH directories (such as ‘$HOME/bin’).

   In Ubuntu (and LilyDev), you can add directories to PATH by adding this line to a hidden file ‘.bashrc’, located in your home directory:

   PATH=~/type-here-directory-containing-git-cl:"${PATH}"
   

git-cl configuration

LilyDev users should perform these ‘configuration’ instructions.

1. You must have a google account; please create one if you do not have one already.

   Note that a google account does not need to be a gmail account; you can use any email address for your google account when you sign up.

2. Move into the top source directory and then configure git cl with the following commands:

   cd $LILYPOND_GIT
   git cl config
   

   For the “Rietveld server” question, the default value (“codereview.appspot.com”) should be accepted by answering with a newline (CR).

   The “CC list” question should be answered with:

   lilypond-devel@gnu.org
   

   The “Tree status URL” value should be left blank. So should the “ViewVC URL” value, since it is used by git cl dcommit which is only for repositories which use git svn (LilyPond doesn’t).

Uploading patch set

There are two methods, depending on your git setup.

• Master branch: (easy option, and used in lily-git.tcl)

  If you added your patch to master, then:

  git pull -r
  git cl upload origin/master
  

  If you have git push ability, make sure that you remove your patch (with git rebase or git reset) before pushing other stuff.

  Notifications of patches are automatically added to our issue tracker to reduce the chance of patches getting lost. To suppress this (not recommended), add the -n / --no-code-issue option.

• Separate branch: (complicated option)

  Ensure your changes are committed in a separate branch, which should differ from the reference branch to be used by just the changes to be uploaded. If the reference branch is to be origin/master, ensure this is up-to-date. If necessary, use git rebase to rebase the branch containing the changes to the head of origin/master. Finally, check out branch with the changes and enter the command:

  git cl upload <reference SHA1 ID>
  

  where <reference SHA1 ID> is the SHA1 ID of the commit to be used as a reference source for the patch. Generally, this will be the SHA1 ID of origin/master, and in that case the command:

  git cl upload origin/master
  

  can be used.

First you will see a terminal editor where you can edit the message that will accompany your patch. git-cl will respect the EDITOR environment variable if defined, otherwise it will use vi as the default editor.

After prompting for your Google email address and password, the patch set will be posted to Rietveld, and you will be given a URL for your patch.

Announcing your patch set

You should then announce the patch by logging into the code review issue webpage and using “Publish + Mail Comments” to add a (mostly bogus) comment to your issue. The text of your comment will be sent to our developer mailing list.

Revisions

As revisions are made in response to comments, successive patch sets for the same issue can be uploaded by reissuing the git-cl command with the modified branch checked out.

Sometimes in response to comments on revisions, the best way to work may require creation of a new branch in git. In order to associate the new branch with an existing Rietveld issue, the following command can be used:

git cl issue issue-number

where issue-number is the number of the existing Rietveld issue.

Resetting git cl

If git cl becomes confused, you can “reset” it by running:

git cl issue 0

Wait for a countdown

Your patch will be available for reviews for the next few hours or days. Three times a week, patches with no known problems are gathered into a “patch countdown” and their status changed to patch-countdown. The countdown is a 48-hour waiting period in which any final reviews or complaints should be made.

During the countdown, your patch may be set to patch-needs_work, indicating that you should fix something (or at least discuss why the patch needs no modification). If no problems are found, the patch will be set to patch-push.

Once a patch has patch-push, it should be sent to your mentor for uploading. If you have git push ability, look at Pushing to staging.

3.4 Advanced Git procedures

It is possible to work with several branches on the same local Git repository; this is especially useful for translators who may have to deal with both translation and a stable branch, e.g. stable/2.12.

Some Git commands are introduced first, then a workflow with several Git branches of LilyPond source code is presented.

3.4.1 Merge conflicts

To be filled in later, and/or moved to a different section. I just wanted to make sure that I had a stub ready somewhere.

3.4.2 Advanced Git concepts

A bit of Git vocabulary will be explained below. The following is only introductory; for a better understanding of Git concepts, you may wish to read Other Git documentation.

The git pull origin command above is just a shortcut for this command:

git pull git://git.sv.gnu.org/lilypond.git/ branch:origin/branch

where branch is typically master or translation; if you do not know or remember, see Downloading remote branches to remember which commands you issued or which source code you wanted to get.

A commit is a set of changes made to the sources; it also includes the committish of the parent commit, the name and e-mail of the author (the person who wrote the changes), the name and e-mail of the committer (the person who brings these changes into the Git repository), and a commit message.

A committish is the SHA1 checksum of a commit, a number made of 40 hexadecimal digits, which acts as the internal unique identifier for this commit. To refer to a particular revision, don’t use vague references like the (approximative) date, simply copy and paste the committish.

A branch is nothing more than a pointer to a particular commit, which is called the head of the branch; when referring to a branch, one often actually thinks about its head and the ancestor commits of the head.

Now we will explain the two last commands you used to get the source code from Git—see Downloading individual branches.

git remote add -ft branch -m branch \
  origin git://git.sv.gnu.org/lilypond.git/

git checkout -b branch origin/branch

The git remote has created a branch called origin/branch in your local Git repository. As this branch is a copy of the remote branch web from git.sv.gnu.org LilyPond repository, it is called a remote branch, and is meant to track the changes on the branch from git.sv.gnu.org: it will be updated every time you run git pull origin or git fetch origin.

The git checkout command has created a branch named branch. At the beginning, this branch is identical to origin/branch, but it will differ as soon as you make changes, e.g. adding newly translated pages or editing some documentation or code source file. Whenever you pull, you merge the changes from origin/branch and branch since the last pulling. If you do not have push (i.e. “write”) access on git.sv.gnu.org, your branch will always differ from origin/branch. In this case, remember that other people working like you with the remote branch branch of git://git.sv.gnu.org/lilypond.git/ (called origin/branch on your local repository) know nothing about your own branch: this means that whenever you use a committish or make a patch, others expect you to take the latest commit of origin/branch as a reference.

Finally, please remember to read the man page of every Git command you will find in this manual in case you want to discover alternate methods or just understand how it works.

3.4.3 Resolving conflicts

Occasionally an update may result in conflicts – this happens when you and somebody else have modified the same part of the same file and git cannot figure out how to merge the two versions together. When this happens, you must manually merge the two versions.

If you need some documentation to understand and resolve conflicts, see paragraphs How conflicts are presented and How to resolve conflicts in git merge man page.

If all else fails, you can follow the instructions in Reverting all local changes. Be aware that this eliminates any changes you have made!

3.4.4 Reverting all local changes

Sometimes git will become hopelessly confused, and you just want to get back to a known, stable state. This command destroys any local changes you have made in the currently checked-out branch, but at least you get back to the current online version:

git reset --hard origin/master

3.4.5 Working with remote branches

Fetching new branches from git.sv.gnu.org

To fetch and check out a new branch named branch on git.sv.gnu.org, run from top of the Git repository

git config --add remote.origin.fetch \
  +refs/heads/branch:refs/remotes/origin/branch

git checkout --track -b branch origin/branch

After this, you can pull branch from git.sv.gnu.org with:

git pull

Note that this command generally fetches all branches you added with git remote add (when you initialized the repository) or git config --add, i.e. it updates all remote branches from remote origin, then it merges the remote branch tracked by the current branch into the current branch. For example, if your current branch is master, origin/master will be merged into master.

Local clones, or having several working trees

If you play with several Git branches, e.g. master, translation, stable/2.12), you may want to have one source and build tree for each branch; this is possible with subdirectories of your local Git repository, used as local cloned subrepositories. To create a local clone for the branch named branch, run

git checkout branch
git clone -lsn . subdir
cd subdir
git reset --hard

Note that subdir must be a directory name which does not already exist. In subdir, you can use all Git commands to browse revisions history, commit and uncommit changes; to update the cloned subrepository with changes made on the main repository, cd into subdir and run git pull; to send changes made on the subrepository back to the main repository, run git push from subdir. Note that only one branch (the currently checked out branch) is created in the subrepository by default; it is possible to have several branches in a subrepository and do usual operations (checkout, merge, create, delete...) on these branches, but this possibility is not detailed here.

When you push branch from subdir to the main repository, and branch is checked out in the main repository, you must save uncommitted changes (see git stash) and do git reset --hard in the main repository in order to apply pushed changes in the working tree of the main repository.

3.4.6 Git log

The commands above don’t only bring you the latest version of the sources, but also the full history of revisions (revisions, also called commits, are changes made to the sources), stored in the ‘.git’ directory. You can browse this history with

git log     # only shows the logs (author, committish and commit message)
git log -p  # also shows diffs
gitk        # shows history graphically

3.4.7 Applying remote patches

TODO: Explain how to determine if a patch was created with git format-patch.

Well-formed git patches created with git format-patch should be committed with the following command:

git am patch

Patches created without git format-patch can be applied in two steps. The first step is to apply the patch to the working tree and the index:

git apply --index patch

The second step is to commit the changes and give credit to the author of the patch. This can be done with the following command:

git commit --author="John Smith <john@example.com>"

Please note that using the --index option for patching is quite important here and cannot reliably be replaced by using the -a option when committing: that would only commit files from the working tree that are already registered with git, so every file that the patch actually adds, like a regtest for a fixed bug, would get lost. For the same reason, you should not use the git-independent ‘patch’ program for applying patches.

3.4.8 Sending and receiving patches via email

The default x-diff MIME type associated with patch files (i.e., files whose name ends in .patch) means that the encoding of line endings may be changed from UNIX to DOS format when they are sent as attachments. Attempting to apply such an inadvertently altered patch will cause git to fail with a message about ‘whitespace errors’.

The solution to such problems is surprisingly simple—just change the default file extension of patches generated by git to end in .txt, for example:

git config format.suffix '.patch.txt'

This should cause email programs to apply the correct base64 encoding to attached patches.

If you receive a patch with DOS instead of UNIX line-endings, it can be converted back using the dos2unix utility.

Lots of useful information on email complications with patches is provided on the Wine wiki at http://wiki.winehq.org/GitWine.

3.4.9 Cleaning up multiple patches

If you have been developing on your own branch for a while, you may have more commmits than is really sensible. To revise your work and condense commits, use:

git rebase origin/master
git rebase -i origin/master

3.4.10 Commit access

Most contributors are not able to commit patches directly to the main repository—only members of the LilyPond development team have commit access. If you are a contributor and are interested in joining the development team, contact the Project Manager through the mailing list (lilypond-devel@gnu.org). Generally, only contributors who have already provided a number of patches which have been pushed to the main repository will be considered for membership.

If you have been approved by the Project Manager, use the following procedure to obtain commit access:

1. If you don’t already have one, set up a Savannah user account at https://savannah.gnu.org/account/register.php. If your web browser responds with an “untrusted connection” message when you visit the link, follow the steps for including the CAcert root certificate in your browser, given at http://savannah.gnu.org/tls/tutorial/.

   Note: Savannah will silently put your username in lower-case – do not try to use capital letters.

2. After registering, if you are not logged in automatically, login at https://savannah.gnu.org/account/login.php—this should take you to your “my” page (https://savannah.gnu.org/my/).
3. Click on the “My Groups” link to access the “My Group Membership” page. From there, find the “Request for Inclusion” box and search for “LilyPond”. Among the search results, check the box labeled “GNU LilyPond Music Typesetter” and write a brief (required) message for the Project Manager (“Hey it’s me!” should be fine).

   Note that you will not have commit access until the Project Manager activates your membership. Once your membership is activated, LilyPond should appear under the heading “Groups I’m Contributor of” on your “My Group Membership” page.

4. Generate an SSH ‘rsa’ key pair. Enter the following at the command prompt:

   ssh-keygen -t rsa
   

   When prompted for a location to save the key, press <ENTER> to accept the default location (‘~/.ssh/id_rsa’).

   Next you are asked to enter an optional passphrase. On most systems, if you use a passphrase, you will likely be prompted for it every time you use git push or git pull. You may prefer this since it can protect you from your own mistakes (like pushing when you mean to pull), though you may find it tedious to keep re-entering it.

   You can change/enable/disable your passphrase at any time with:

   ssh-keygen -f ~/.ssh/id_rsa -p
   

   Note that the GNOME desktop has a feature which stores your passphrase for you for an entire GNOME session. If you use a passphrase to “protect you from yourself”, you will want to disable this feature, since you’ll only be prompted once. Run the following command, then logout of GNOME and log back in:

   gconftool-2 --set -t bool \
     /apps/gnome-keyring/daemon-components/ssh false
   

   After setting up your passphrase, your private key is saved as ‘~/.ssh/id_rsa’ and your public key is saved as ‘~/.ssh/id_rsa.pub’.

5. Register your public SSH ‘rsa’ key with Savannah. From the “My Account Configuration” page, click on “Edit SSH Keys”, then paste the contents of your ‘~/.ssh/id_rsa.pub’ file into one of the “Authorized keys” text fields, and click “Update”.

   Savannah should respond with something like:

   Success: Key #1 seen Keys registered
   

6. Configure Git to use the SSH protocol (instead of the GIT protocol). From your local Git repository, enter:

   git config remote.origin.url \
     ssh://user@git.sv.gnu.org/srv/git/lilypond.git
   

   replacing user with your Savannah username.

7. After your membership has been activated and you’ve configured Git to use SSH, test the connection with:

   git pull --verbose
   

   SSH should issue the following warning:

   The authenticity of host 'git.sv.gnu.org (140.186.70.72)' can't
   be established.
   RSA key fingerprint is
   80:5a:b0:0c:ec:93:66:29:49:7e:04:2b:fd:ba:2c:d5.
   Are you sure you want to continue connecting (yes/no)?
   

   Make sure the RSA key fingerprint displayed matches the one above. If it doesn’t, respond “no” and check that you configured Git properly in the previous step. If it does match, respond “yes”. SSH should then issue another warning:

   Warning: Permanently added 'git.sv.gnu.org,140.186.70.72' (RSA) to
   the list of known hosts.
   

   The list of known hosts is stored in the file ‘~/.ssh/known_hosts’.

   At this point, you are prompted for your passphrase if you have one, then Git will attempt a pull.

   If git pull --verbose fails, you should see error messages like these:

   Permission denied (publickey).
   fatal: The remote end hung up unexpectedly
   

   If you get the above error, you may have made a mistake when registering your SSH key at Savannah. If the key is properly registered, you probably just need to wait for the Savannah server to activate it. It usually takes a few minutes for the key to be active after registering it, but if it still doesn’t work after an hour, ask for help on the mailing list.

   If git pull --verbose succeeds, the output will include a ‘From’ line that shows ‘ssh’ as the protocol:

   From ssh://git.sv.gnu.org/srv/git/lilypond
   

   If the protocol shown is not ‘ssh’, check that you configured Git properly in the previous step.

8. Test your commit access with a dry run:

   Note: Do not push directly to master; instead, push to staging. See Pushing to staging.

   git push --dry-run --verbose
   

   Note that recent versions of Git (Git 1.6.3 or later) will issue a big warning if the above command is used. The simplest solution is to tell Git to push all matching branches by default:

   git config push.default matching
   

   Then git push should work as before. For more details, consult the git push man page.

9. Repeat the steps from generating an RSA key through to testing your commit access, for each machine from which you will be making commits, or you may simply copy the files from your local ‘~/.ssh’ folder to the same folder on the other machine.

Technical details

• On Firefox, to view or remove the CAcert root certificate, go to: Edit > Preferences > Advanced > Encryption > View Certificates > Authorities > Certificate Name > Root CA > CA Cert Signing Authority.
• The git config commands above should modify your local repository’s ‘.git/config’ file. These lines:

  [remote "origin"]
          url = git://git.sv.gnu.org/lilypond.git/
  

  should now be changed to:

  [remote "origin"]
          url = ssh://user@git.sv.gnu.org/srv/git/lilypond.git
  

  where user is your login name on Savannah.

• Similarly, the git config push.default matching command should add these lines to ‘.git/config’:

  [push]
          default = matching
  

Known issues and warnings

Encryption protocols, including ssh, generally do not permit packet fragmentation to avoid introducing a point of insecurity. This means that the maximum packet size must not exceed the smallest MTU (Maximum Transmission Unit) set in the routers along the path. This smallest MTU is determined by a procedure during call set-up which relies on the transmission over the path of ICMP packets. If any of the routers in the path block ICMP packets this mechanism fails, resulting in the possibility of packets being transmitted which exceed the MTU of one of the routers. If this happens the packet is discarded, causing the ssh session to hang, timeout or terminate with the error message

ssh: connect to host <host ip addr> port 22: Bad file number
fatal: The remote end hung up unexpectedly

depending on precisely when in the proceedings the first large packet is transmitted. Most routers on the internet have MTU set to 1500, but routers installed in homes to connect via broadband may use a slightly smaller MTU for efficient transmission over ATM. If this problem is encountered a possible work-around is to set the MTU in the local router to 1500.

3.4.11 Pushing to staging

Do not push directly to the git master branch. Instead, push to staging.

You will not see your patch on origin/master until some automatic tests have been run. These tests are run every couple of hours; please wait at least 12 hours before wondering if your patch has been lost. Note that you can check the commits on origin/staging by looking at the git web interface on savannah.

It may happen occasionally that the staging branch breaks automated testing. In this case the automatic move of staging material to master gets halted in order to avoid broken material entering master. This is a safety net. Please do not try breaking out from it by adding fixes on top of staging: in that case the whole sequence will end up in master after all, defeating the purpose of the system. The proper fix usually involves rewriting the staging branch and is best left to core developers after discussion on the developer list.

If your work is in a patch file

Assuming that your patch is in a file called ‘0001-my-patch.patch’, and you are currently on git master, do:

git checkout staging
git pull -r
git am 0001-my-patch.patch
gitk
git push origin staging
git checkout master

If your work is in a branch

If you are working on branches and your work in is my_branch_name, then do:

git checkout staging
git pull -r
git merge my_branch_name
gitk
git push origin staging

3.5 Git on Windows

TODO: Decide what to do with this... Pare it down? Move paragraphs next to analogous Unix instructions? -mp

3.5.1 Background to nomenclature

Git is a system for tracking the changes made to source files by a distributed set of editors. It is designed to work without a master repository, but we have chosen to have a master repository for LilyPond files. Editors hold a local copy of the master repository together with any changes they have made locally. Local changes are held in a local ‘branch’, of which there may be several, but these instructions assume you are using just one. The files visible in the local repository always correspond to those on the currently ‘checked out’ local branch.

Files are edited on a local branch, and in that state the changes are said to be ‘unstaged’. When editing is complete, the changes are moved to being ‘staged for commit’, and finally the changes are ‘committed’ to the local branch. Once committed, the changes (called a ‘commit’) are given a unique 40-digit hexadecimal reference number called the ‘Committish’ or ‘SHA1 ID’ which identifies the commit to Git. Such committed changes can be sent to the master repository by ‘pushing’ them (if you have write permission) or by sending them by email to someone who has, either as a complete file or as a ‘diff’ or ‘patch’ (which send just the differences from the master repository).

3.5.2 Installing git

Obtain Git from http://code.google.com/p/msysgit/downloads/list (note, not msysGit, which is for Git developers and not PortableGit, which is not a full git installation) and install it.

Note that most users will not need to install SSH. That is not required until you have been granted direct push permissions to the master git repository.

Start Git by clicking on the desktop icon. This will bring up a command line bash shell. This may be unfamiliar to Windows users. If so, follow these instructions carefully. Commands are entered at a $ prompt and are terminated by keying a newline.

3.5.3 Initialising Git

Decide where you wish to place your local Git repository, creating the folders in Windows as necessary. Here we call the folder to contain the repository [path]/Git, but if you intend using Git for other projects a directory name like lilypond-git might be better. You will need to have space for around 100Mbytes.

Start the Git bash shell by clicking on the desk-top icon installed with Git and type

cd [path]/Git

to position the shell at your new Git repository.

Note: if [path] contains folders with names containing spaces use

cd "[path]/Git"

Then type

git init

to initialize your Git repository.

Then type (all on one line; the shell will wrap automatically)

git remote add -ft master origin git://git.sv.gnu.org/lilypond.git

to download the lilypond master files.

We now need to generate a local copy of the downloaded files in a new local branch. Your local branch needs to have a name. It is usual to call it ‘master’ and we shall do that here.

To do this, type

git checkout -b master origin/master

This creates a second branch called ‘master’. You will see two warnings (ignore these), and a message advising you that your local branch ‘master’ has been set up to track the remote branch. You now have two branches, a local branch called ‘master’, and a tracking branch called ‘origin/master’, which is a shortened form of ‘remotes/origin/master’.

Return to Windows Explorer and look in your Git repository. You should see lots of folders. For example, the LilyPond documentation can be found in [path]/Git/Documentation/.

The Git bash shell is terminated by typing exit or by clicking on the usual Windows close-window widget.

3.5.4 Git GUI

Almost all subsequent work will use the Git Graphical User Interface, which avoids having to type command line commands. To start Git GUI first start the Git bash shell by clicking on the desktop icon, and type

cd [path]/Git
git gui

The Git GUI will open in a new window. It contains four panels and 7 pull-down menus. At this stage do not use any of the commands under Branch, Commit, Merge or Remote. These will be explained later.

The top panel on the left contains the names of files which you are in the process of editing (Unstaged Changes), and the lower panel on the left contains the names of files you have finished editing and have staged ready for committing (Staged Changes). At present, these panels will be empty as you have not yet made any changes to any file. After a file has been edited and saved the top panel on the right will display the differences between the edited file selected in one of the panels on the left and the last version committed on the current branch.

The panel at bottom right is used to enter a descriptive message about the change before committing it.

The Git GUI is terminated by entering CNTL-Q while it is the active window or by clicking on the usual Windows close-window widget.

3.5.5 Personalising your local git repository

Open the Git GUI, click on

Edit -> Options

and enter your name and email address in the left-hand (Git Repository) panel. Leave everything else unchanged and save it.

Note that Windows users must leave the default setting for line endings unchanged. All files in a git repository must have lines terminated by just a LF, as this is required for Merge to work, but Windows files are terminated by CRLF by default. The git default setting causes the line endings of files in a Windows git repository to be flipped automatically between LF and CRLF as required. This enables files to be edited by any Windows editor without causing problems in the git repository.

3.5.6 Checking out a branch

At this stage you have two branches in your local repository, both identical. To see them click on

Branch -> Checkout

You should have one local branch called ‘master’ and one tracking branch called ‘origin/master’. The latter is your local copy of the ‘remotes/origin/master’ branch in the master LilyPond repository. The local ‘master’ branch is where you will make your local changes.

When a particular branch is selected, i.e., checked out, the files visible in your repository are changed to reflect the state of the files on that branch.

3.5.7 Updating files from ‘remote/origin/master’

Before starting the editing of a file, ensure your local repository contains the latest version of the files in the remote repository by first clicking

Remote -> Fetch from -> origin

in the Git GUI.

This will place the latest version of every file, including all the changes made by others, into the ‘origin/master’ branch of the tracking branches in your git repository. You can see these files by checking out this branch, but you must never edit any files while this branch is checked out. Check out your local ‘master’ branch again.

You then need to merge these fetched files into your local ‘master’ branch by clicking on

Merge -> Local Merge

and if necessary select the local ‘master’ branch.

Note that a merge cannot be completed if you have made any local changes which have not yet been committed.

This merge will update all the files in the ‘master’ branch to reflect the current state of the ‘origin/master’ branch. If any of the changes conflict with changes you have made yourself recently you will be notified of the conflict (see below).

3.5.8 Editing files

First ensure your ‘master’ branch is checked out, then simply edit the files in your local Git repository with your favourite editor and save them back there. If any file contains non-ASCII characters ensure you save it in UTF-8 format. Git will detect any changes whenever you restart Git GUI and the file names will then be listed in the Unstaged Changes panel. Or you can click the Rescan button to refresh the panel contents at any time. You may break off and resume editing any time.

The changes you have made may be displayed in diff form in the top right-hand panel of Git GUI by clicking on the file name shown in one of the left panels.

When your editing is complete, move the files from being Unstaged to Staged by clicking the document symbol to the left of each name. If you change your mind it can be moved back by clicking on the ticked box to the left of the name.

Finally the changes you have made may be committed to your ‘master’ branch by entering a brief message in the Commit Message box and clicking the Commit button.

If you wish to amend your changes after a commit has been made, the original version and the changes you made in that commit may be recovered by selecting

Commit -> Amend Last Commit

or by checking the Amend Last Commit radio button at bottom right. This will return the changes to the Staged state, so further editing made be carried out within that commit. This must only be done before the changes have been Pushed or sent to your mentor for Pushing - after that it is too late and corrections have to be made as a separate commit.

3.5.9 Sending changes to ‘remotes/origin/master’

If you do not have write access to ‘remotes/origin/master’ you will need to send your changes by email to someone who does.

First you need to create a diff or patch file containing your changes. To create this, the file must first be committed. Then terminate the Git GUI. In the git bash shell first cd to your Git repository with

cd [path]/Git

if necessary, then produce the patch with

git format-patch origin

This will create a patch file for all the locally committed files which differ from ‘origin/master’. The patch file can be found in [path]/Git and will have a name formed from the commit message.

3.5.10 Resolving merge conflicts

As soon as you have committed a changed file your local master branch has diverged from origin/master, and will remain diverged until your changes have been committed in remotes/origin/master and Fetched back into your origin/master branch. Similarly, if a new commit has been made to remotes/origin/master by someone else and Fetched, your local master branch is divergent. You can detect a divergent branch by clicking on

Repository -> Visualise all branch history

This opens up a very useful new window called ‘gitk’. Use this to browse all the commits made by yourself and others.

If the diagram at top left of the resulting window does not show your master tag on the same node as the remotes/origin/master tag your branch has diverged from origin/master. This is quite normal if files you have modified yourself have not yet been Pushed to remotes/origin/master and Fetched, or if files modified and committed by others have been Fetched since you last Merged origin/master into your local master branch.

If a file being merged from origin/master differs from one you have modified in a way that cannot be resolved automatically by git, Merge will report a Conflict which you must resolve by editing the file to create the version you wish to keep.

This could happen if the person updating remotes/origin/master for you has added some changes of his own before committing your changes to remotes/origin/master, or if someone else has changed the same file since you last fetched the file from remotes/origin/master.

Open the file in your editor and look for sections which are delimited with ...

[to be completed when I next have a merge conflict to be sure I give the right instructions -td]

3.5.11 Other actions

The instructions above describe the simplest way of using git on Windows. Other git facilities which may usefully supplement these include

• Using multiple local branches (Create, Rename, Delete)
• Resetting branches
• Cherry-picking commits
• Pushing commits to remote/origin/master
• Using gitk to review history

Once familiarity with using git on Windows has been gained the standard git manuals can be used to learn about these.

3.6 Repository directory structure

Prebuilt Documentation and packages are available from:

    http://www.lilypond.org

LilyPond development is hosted at:

    http://savannah.gnu.org/projects/lilypond

Here is a simple explanation of the directory layout for
LilyPond's source files.


.                        Toplevel READMEs, ChangeLog,
|                          build bootstrapping, patches
|                          for third party programs
|
|-- Documentation/       Top sources for manuals
|   |
|   |
|   |   INDIVIDUAL CHAPTERS FOR EACH MANUAL:
|   |
|   |-- contributor/     Contributor's Guide
|   |-- essay/           Essay on automated music engraving
|   |-- extending/       Extending
|   |-- learning/        Learning Manual
|   |-- notation/        Notation Reference
|   |-- usage/           Usage
|   |-- web/             The website
|   |
|   |
|   |   TRANSLATED MANUALS:
|   |     Each language's directory can contain...
|   |       1) translated versions of:
|   |          * top sources for manuals
|   |          * individual chapters for each manual
|   |       2) a texidocs/ directory for snippet translations
|   |
|   |-- cs/              Czech
|   |-- de/              German
|   |-- es/              Spanish
|   |-- fr/              French
|   |-- hu/              Hungarian
|   |-- it/              Italian
|   |-- ja/              Japanese
|   |-- nl/              Dutch
|   |-- zh/              Chinese
|   |
|   |
|   |   MISCELLANEOUS DOC STUFF:
|   |
|   |-- css/             CSS files for HTML docs
|   |-- included/        .ly files used in the manuals
|   |-- logo/            Web logo and "note" icon
|   |-- ly-examples/     .ly files for the "Examples" webpage
|   |-- misc/            Old announcements, ChangeLogs and NEWS
|   |-- pictures/        Images used (eps/jpg/png/svg)
|   |   `-- pdf/         (pdf)
|   |-- po/              Translated build/maintenance scripts
|   |-- snippets/        Auto-generated .ly snippets (from the LSR)
|   |   `-- new/         Snippets too new for the LSR
|   `-- topdocs/         AUTHORS, INSTALL, README
|
|
|   C++ SOURCES:
|
|-- flower/              A simple C++ library
|-- lily/                C++ sources for the LilyPond binary
|
|
|   LIBRARIES:
|
|-- ly/                  .ly \include files
|-- mf/                  MetaFont sources for Emmentaler fonts
|-- ps/                  PostScript library files
|-- scm/                 Scheme sources for LilyPond and subroutine files
|-- tex/                 TeX and texinfo library files
|
|
|   SCRIPTS:
|
|-- config/              Autoconf helpers for configure script
|-- python/              Python modules, MIDI module
|   `-- auxiliar/        Python modules for build/maintenance
|-- scripts/             End-user scripts (--> lilypond/usr/bin/)
|   |-- auxiliar/        Maintenance and non-essential build scripts
|   `-- build/           Essential build scripts
|
|
|   BUILD PROCESS:
|   (also see SCRIPTS section above)
|
|-- make/                Specific make subroutine files
|-- stepmake/            Generic make subroutine files
|
|
|   REGRESSION TESTS:
|
|-- input/
|   `-- regression/      .ly regression tests
|       |-- abc2ly/      .abc regression tests
|       |-- lilypond-book/  lilypond-book regression tests
|       |-- midi/        midi2ly regression tests
|       `-- musicxml/    .xml and .itexi regression tests
|
|
|   MISCELLANEOUS:
|
|-- elisp/               Emacs LilyPond mode and syntax coloring
|-- vim/                 Vi(M) LilyPond mode and syntax coloring
`-- po/                  Translations for binaries and end-user scripts

3.7 Other Git documentation

• Official git man pages: http://www.kernel.org/pub/software/scm/git/docs/
• More in-depth tutorials: http://git-scm.com/documentation
• Book about git: Pro Git
• Github help: http://help.github.com/ (very highly recommended by Graham)

4. Compiling

This chapter describes the process of compiling the LilyPond program from source files.

4.1 Overview of compiling

Compiling LilyPond from source is an involved process, and is only recommended for developers and packagers. Typical program users are instead encouraged to obtain the program from a package manager (on Unix) or by downloading a precompiled binary configured for a specific operating system. Pre-compiled binaries are available on the Download page.

Compiling LilyPond from source is necessary if you want to build, install, or test your own version of the program.

A successful compile can also be used to generate and install the documentation, incorporating any changes you may have made. However, a successful compile is not a requirement for generating the documentation. The documentation can be built using a Git repository in conjunction with a locally installed copy of the program. For more information, see Building documentation without compiling.

Attempts to compile LilyPond natively on Windows have been unsuccessful, though a workaround is available (see LilyDev).

4.2 Requirements

4.2.1 Requirements for running LilyPond

Running LilyPond requires proper installation of the following software:

• DejaVu fonts (normally installed by default)
• FontConfig (2.4.0 or newer)
• Freetype (2.1.10 or newer)
• Ghostscript (8.60 or newer)
• Guile (1.8.2 or newer)
• Pango (1.12 or newer)
• Python (2.4 or newer)

International fonts are required to create music with international text or lyrics.

4.2.2 Requirements for compiling LilyPond

Below is a full list of packages needed to build LilyPond. However, for most common distributions there is an easy way of installing most all build dependencies in one go:

• Everything listed in Requirements for running LilyPond
• Development packages for the above items (which should include header files and libraries).

  Red Hat Fedora:

  guile-devel-version
  fontconfig-devel-version
  freetype-devel-version
  pango-devel-version
  python-devel-version
  

  Debian GNU/Linux:

  guile-version-dev
  libfontconfig1-dev
  libfreetype6-dev
  libpango1.0-dev
  pythonversion-dev
  

• Flex
• FontForge (20060125 or newer; 20100501 or newer is recommended; must be compiled with ‘--enable-double’. Failure to do so can lead to poor intersection calculations and poorly-rendered glyphs.)
• GNU Bison
• GNU Compiler Collection (3.4 or newer, 4.x recommended)
• GNU gettext (0.17 or newer)
• GNU Make (3.78 or newer)
• MetaFont (mf-nowin, mf, mfw or mfont binaries), usually packaged with TeX.
• MetaPost (mpost binary), usually packaged with TeX.
• Perl
• Texinfo (4.11 or newer)
• Type 1 utilities (1.33 or newer recommended)

4.2.3 Requirements for building documentation

You can view the documentation online at http://www.lilypond.org/doc/, but you can also build it locally. This process requires some additional tools and packages:

• Everything listed in Requirements for compiling LilyPond
• ImageMagick
• Netpbm
• gzip
• rsync
• Texi2HTML (1.82)
• International fonts

  Red Hat Fedora:

  fonts-arabic
  fonts-hebrew
  fonts-ja
  fonts-xorg-truetype
  taipeifonts
  ttfonts-ja
  ttfonts-zh_CN
  

  Debian GNU/Linux:

  emacs-intl-fonts
  ttf-kochi-gothic
  ttf-kochi-mincho
  xfonts-bolkhov-75dpi
  xfonts-cronyx-75dpi
  xfonts-cronyx-100dpi
  xfonts-intl-.*
  

4.3 Getting the source code

Downloading the Git repository

In general, developers compile LilyPond from within a local Git repository. Setting up a local Git repository is explained in Starting with Git.

Downloading a source tarball

Packagers are encouraged to use source tarballs for compiling.

The tarball for the latest stable release is available on the Source page.

The latest source code snapshot is also available as a tarball from the GNU Savannah Git server.

All tagged releases (including legacy stable versions and the most recent development release) are available here:

http://download.linuxaudio.org/lilypond/source/

Download the tarball to your ‘~/src/’ directory, or some other appropriate place.

Unpack the tarball with this command:

tar -xzf lilypond-x.y.z.tar.gz

This creates a subdirectory within the current directory called ‘lilypond-x.y.z/’. Once unpacked, the source files occupy about 40 MB of disk space.

Windows users wanting to look at the source code may have to download and install the free-software 7zip archiver to extract the tarball.

4.4 Configuring make

4.4.1 Running ./autogen.sh

After you unpack the tarball (or download the Git repository), the contents of your top source directory should be similar to the current source tree listed at http://git.sv.gnu.org/gitweb/?p=lilypond.git;a=tree.

Next, you need to create the generated files; enter the following command from your top source directory:

./autogen.sh --noconfigure

This will generate a number of files and directories to aid configuration, such as ‘configure’, ‘README.txt’, etc.

Next, create the build directory with:

mkdir build/
cd build/

We heavily recommend building lilypond inside a separate directory with this method.

4.4.2 Running ../configure

Configuration options

The ../configure command (generated by ./autogen.sh) provides many options for configuring make. To see them all, run:

../configure --help

Checking build dependencies

When ../configure is run without any arguments, it will check to make sure your system has everything required for compilation:

../configure

If any build dependency is missing, ../configure will return with:

ERROR: Please install required programs:  foo

The following message is issued if you are missing programs that are only needed for building the documentation:

WARNING: Please consider installing optional programs:  bar

If you intend to build the documentation locally, you will need to install or update these programs accordingly.

Configuring target directories

If you intend to use your local build to install a local copy of the program, you will probably want to configure the installation directory. Here are the relevant lines taken from the output of ../configure --help:

By default, ‘make install’ will install all the files in ‘/usr/local/bin’, ‘/usr/local/lib’ etc. You can specify an installation prefix other than ‘/usr/local’ using ‘‘--prefix’’, for instance ‘‘--prefix=$HOME’’.

A typical installation prefix is ‘$HOME/usr’:

../configure --prefix=$HOME/usr

Note that if you plan to install a local build on a system where you do not have root privileges, you will need to do something like this anyway—make install will only succeed if the installation prefix points to a directory where you have write permission (such as your home directory). The installation directory will be automatically created if necessary.

The location of the lilypond command installed by this process will be ‘prefix/bin/lilypond’; you may want to add ‘prefix/bin/’ to your $PATH if it is not already included.

It is also possible to specify separate installation directories for different types of program files. See the full output of ../configure --help for more information.

If you encounter any problems, please see Problems.

4.5 Compiling LilyPond

4.5.1 Using make

LilyPond is compiled with the make command. Assuming make is configured properly, you can simply run:

make

‘make’ is short for ‘make all’. To view a list of make targets, run:

make help

TODO: Describe what make actually does.

See also

Generating documentation provides more info on the make targets used to build the LilyPond documentation.

4.5.2 Saving time with the ‘-j’ option

If your system has multiple CPUs, you can speed up compilation by adding ‘-jX’ to the make command, where ‘X’ is one more than the number of cores you have. For example, a typical Core2Duo machine would use:

make -j3

If you get errors using the ‘-j’ option, and ‘make’ succeeds without it, try lowering the X value.

Because multiple jobs run in parallel when ‘-j’ is used, it can be difficult to determine the source of an error when one occurs. In that case, running ‘make’ without the ‘-j’ is advised.

4.5.3 Compiling for multiple platforms

If you want to build multiple versions of LilyPond with different configuration settings, you can use the ‘--enable-config=conf’ option of configure. You should use make conf=conf to generate the output in ‘out-conf’. For example, suppose you want to build with and without profiling, then use the following for the normal build

./configure --prefix=$HOME/usr/ --enable-checking
make

and for the profiling version, specify a different configuration

./configure --prefix=$HOME/usr/ --enable-profiling \
  --enable-config=prof --disable-checking
make conf=prof

If you wish to install a copy of the build with profiling, don’t forget to use conf=CONF when issuing make install:

make conf=prof install

See also

Installing LilyPond from a local build

4.5.4 Useful make variables

If a less verbose build output if desired, the variable QUIET_BUILD may be set to 1 on make command line, or in ‘local.make’ at top of the build tree.

4.6 Post-compilation options

4.6.1 Installing LilyPond from a local build

If you configured make to install your local build in a directory where you normally have write permission (such as your home directory), and you have compiled LilyPond by running make, you can install the program in your target directory by running:

make install

If instead, your installation directory is not one that you can normally write to (such as the default ‘/usr/local/’, which typically is only writeable by the superuser), you will need to temporarily become the superuser when running make install:

sudo make install

or…

su -c 'make install'

If you don’t have superuser privileges, then you need to configure the installation directory to one that you can write to, and then re-install. See Configuring target directories.

4.6.2 Generating documentation

Documentation editor’s edit/compile cycle

• Initial documentation build:

  make [-jX]
  make [-jX CPU_COUNT=X] doc          ## can take an hour or more
  make [-jX CPU_COUNT=X] doc-stage-1  ## to build only PDF documentation
  

• Edit/compile cycle:

  ## edit source files, then…
  
  make [-jX]                  ## needed if editing outside
                              ##   Documentation/, but useful anyway
                              ##   for finding Texinfo errors.
  make [-jX CPU_COUNT=X] doc  ## usually faster than initial build.
  

• Reset:

  It is generally possible to remove the compiled documentation from your system with ‘make doc-clean’, but this method is not 100% guaranteed. Instead, if you want to be sure you have a clean system, we recommend that you delete your ‘build/’ directory, and begin compiling from scratch. Since the documentation compile takes much longer than the non-documentation compile, this does not increase the overall time by a great deal.

Building documentation

After a successful compile (using make), the documentation can be built by issuing:

make doc

or, to build only the PDF documentation and not the HTML,

make doc-stage-1

After this initial build, make doc only makes changes to the documentation where needed, so it may only take a minute or two to test changes if the documentation is already built.

If make doc succeeds, the HTML documentation tree is available in ‘out-www/offline-root/’, and can be browsed locally. Various portions of the documentation can be found by looking in ‘out/’ and ‘out-www’ subdirectories in other places in the source tree, but these are only portions of the docs. Please do not complain about anything which is broken in those places; the only complete set of documentation is in ‘out-www/offline-root/’ from the top of the source tree.

make doc sends the output from most of the compilation to logfiles. If the build fails for any reason, it should prompt you with the name of a logfile which will provide information to help you work out why the build failed. These logfiles are not deleted with make doc-clean. To remove all the logfiles generated by the compilation process, use:

make log-clean

make doc compiles the documents for all languages. To save some compile time, the English language documents can be compiled on their own with:

make LANGS='' doc

Similarly, it is possible to compile a subset of the translated documentation by specifying their language codes on the command line. For example, the French and German translations are compiled with:

make LANGS='de fr' doc

Note that this will also compile the English version.

Compilation of documentation in Info format with images can be done separately by issuing:

make info

An issue when switching branches between master and translation is the appearance/disappearance of translated versions of some manuals. If you see such a warning from make:

No rule to make target `X', needed by `Y'

Your best bet is to delete the file Y.dep and to try again.

Building a single document

It’s possible to build a single document. For example, to rebuild only ‘contributor.pdf’, do the following:

cd build/
cd Documentation/
touch ../../Documentation/contributor.texi
make out=www out-www/contributor.pdf

If you are only working on a single document, test-building it in this way can give substantial time savings - recreating ‘contributor.pdf’, for example, takes a matter of seconds.

Saving time with CPU_COUNT

The most time consuming task for building the documentation is running LilyPond to build images of music, and there cannot be several simultaneously running lilypond-book instances, so the ‘-j’ make option does not significantly speed up the build process. To help speed it up, the makefile variable ‘CPU_COUNT’ may be set in ‘local.make’ or on the command line to the number of .ly files that LilyPond should process simultaneously, e.g. on a bi-processor or dual core machine:

make -j3 CPU_COUNT=3 doc

The recommended value of ‘CPU_COUNT’ is one plus the number of cores or processors, but it is advisable to set it to a smaller value unless your system has enough RAM to run that many simultaneous LilyPond instances. Also, values for the ‘-j’ option that pose problems with ‘make’ are less likely to pose problems with ‘make doc’ (this applies to both ‘-j’ and ‘CPU_COUNT’). For example, with a quad-core processor, it is possible for ‘make -j5 CPU_COUNT=5 doc’ to work consistently even if ‘make -j5’ rarely succeeds.

AJAX search

To build the documentation with interactive searching, use:

make doc AJAX_SEARCH=1

This requires PHP, and you must view the docs via a http connection (you cannot view them on your local filesystem).

Installing documentation

The HTML, PDF and if available Info files can be installed into the standard documentation path by issuing

make install-doc

This also installs Info documentation with images if the installation prefix is properly set; otherwise, instructions to complete proper installation of Info documentation are printed on standard output.

To install the Info documentation separately, run:

make install-info

Note that to get the images in Info documentation, install-doc target creates symbolic links to HTML and PDF installed documentation tree in ‘prefix/share/info’, in order to save disk space, whereas install-info copies images in ‘prefix/share/info’ subdirectories.

It is possible to build a documentation tree in ‘out-www/online-root/’, with special processing, so it can be used on a website with content negotiation for automatic language selection; this can be achieved by issuing

make WEB_TARGETS=online doc

and both ‘offline’ and ‘online’ targets can be generated by issuing

make WEB_TARGETS="offline online" doc

Several targets are available to clean the documentation build and help with maintaining documentation; an overview of these targets is available with

make help

from every directory in the build tree. Most targets for documentation maintenance are available from ‘Documentation/’; for more information, see Documentation work.

The makefile variable QUIET_BUILD may be set to 1 for a less verbose build output, just like for building the programs.

Building documentation without compiling

The documentation can be built locally without compiling LilyPond binary, if LilyPond is already installed on your system.

From a fresh Git checkout, do

./autogen.sh   # ignore any warning messages
cp GNUmakefile.in GNUmakefile
make -C scripts && make -C python
nice make LILYPOND_EXTERNAL_BINARY=/path/to/bin/lilypond doc

Please note that this may break sometimes – for example, if a new feature is added with a test file in input/regression, even the latest development release of LilyPond will fail to build the docs.

You may build the manual without building all the ‘input/*’ stuff (i.e. mostly regression tests): change directory, for example to ‘Documentation/’, issue make doc, which will build documentation in a subdirectory ‘out-www’ from the source files in current directory. In this case, if you also want to browse the documentation in its post-processed form, change back to top directory and issue

make out=www WWW-post

Known issues and warnings

You may also need to create a script for pngtopnm and pnmtopng. On GNU/Linux, I use this:

export LD_LIBRARY_PATH=/usr/lib
exec /usr/bin/pngtopnm "$@"

On MacOS X with fink, I use this:

export DYLD_LIBRARY_PATH=/sw/lib
exec /sw/bin/pngtopnm "$@"

On MacOS X with macports, you should use this:

export DYLD_FALLBACK_LIBRARY_PATH=/opt/local/lib
exec /opt/local/bin/pngtopnm "$@"

4.6.3 Testing LilyPond binary

LilyPond comes with an extensive suite that exercises the entire program. This suite can be used to test that the binary has been built correctly.

The test suite can be executed with:

make test

If the test suite completes successfully, the LilyPond binary has been verified.

More information on the regression test suite is found at Regression tests.

4.7 Problems

For help and questions use lilypond-user@gnu.org. Send bug reports to bug-lilypond@gnu.org.

Bugs that are not fault of LilyPond are documented here.

Bison 1.875

There is a bug in bison-1.875: compilation fails with "parse error before ‘goto’" in line 4922 due to a bug in bison. To fix, please recompile bison 1.875 with the following fix

$ cd lily; make out/parser.cc
$ vi +4919 out/parser.cc
# append a semicolon to the line containing "__attribute__ ((__unused__))
# save
$ make

Compiling on MacOS X

Here are special instructions for compiling under MacOS X. These instructions assume that dependencies are installed using MacPorts. The instructions have been tested using OS X 10.5 (Leopard).

First, install the relevant dependencies using MacPorts.

Next, add the following to your relevant shell initialization files. This is ~/.profile by default. You should create this file if it does not exist.

export PATH=/opt/local/bin:/opt/local/sbin:$PATH
export DYLD_FALLBACK_LIBRARY_PATH=/opt/local/lib:$DYLD_FALLBACK_LIBRARY_PATH

Now you must edit the generated ‘config.make’ file. Change

FLEXLEXER_FILE = /usr/include/FlexLexer.h

to:

FLEXLEXER_FILE = /opt/local/include/FlexLexer.h

At this point, you should verify that you have the appropriate fonts installed with your ghostscript installation. Check ls /opt/local/share/ghostscript/fonts for: ’c0590*’ files (.pfb, .pfb and .afm). If you don’t have them, run the following commands to grab them from the ghostscript SVN server and install them in the appropriate location:

svn export http://svn.ghostscript.com/ghostscript/tags/urw-fonts-1.0.7pre44/
sudo mv urw-fonts-1.0.7pre44/* /opt/local/share/ghostscript/fonts/
rm -rf urw-fonts-1.07pre44

Now run the ./configure script. To avoid complications with automatic font detection, add

--with-ncsb-dir=/opt/local/share/ghostscript/fonts

Solaris

Solaris7, ./configure

‘./configure’ needs a POSIX compliant shell. On Solaris7, ‘/bin/sh’ is not yet POSIX compliant, but ‘/bin/ksh’ or bash is. Run configure like

CONFIG_SHELL=/bin/ksh ksh -c ./configure

or

CONFIG_SHELL=/bin/bash bash -c ./configure

FreeBSD

To use system fonts, dejaview must be installed. With the default port, the fonts are installed in ‘usr/X11R6/lib/X11/fonts/dejavu’.

Open the file ‘$LILYPONDBASE/usr/etc/fonts/local.conf’ and add the following line just after the <fontconfig> line. (Adjust as necessary for your hierarchy.)

<dir>/usr/X11R6/lib/X11/fonts</dir>

International fonts

On Mac OS X, all fonts are installed by default. However, finding all system fonts requires a bit of configuration; see this post on the lilypond-user mailing list.

On Linux, international fonts are installed by different means on every distribution. We cannot list the exact commands or packages that are necessary, as each distribution is different, and the exact package names within each distribution changes. Here are some hints, though:

Red Hat Fedora

    taipeifonts fonts-xorg-truetype ttfonts-ja fonts-arabic \
         ttfonts-zh_CN fonts-ja fonts-hebrew

Debian GNU/Linux

   apt-get install emacs-intl-fonts xfonts-intl-.* \
        ttf-kochi-gothic ttf-kochi-mincho \
        xfonts-bolkhov-75dpi xfonts-cronyx-100dpi xfonts-cronyx-75dpi

Using lilypond python libraries

If you want to use lilypond’s python libraries (either running certain build scripts manually, or using them in other programs), set PYTHONPATH to ‘python/out’ in your build directory, or ‘…/usr/lib/lilypond/current/python’ in the installation directory structure.

4.8 Concurrent stable and development versions

It can be useful to have both the stable and the development versions of Lilypond available at once. One way to do this on GNU/Linux is to install the stable version using the precompiled binary, and run the development version from the source tree. After running make all from the top directory of the Lilypond source files, there will be a binary called lilypond in the out directory:

<path to>/lilypond/out/bin/lilypond

This binary can be run without actually doing the make install command. The advantage to this is that you can have all of the latest changes available after pulling from git and running make all, without having to uninstall the old version and reinstall the new.

So, to use the stable version, install it as usual and use the normal commands:

lilypond foobar.ly

To use the development version, create a link to the binary in the source tree by saving the following line in a file somewhere in your $PATH:

exec <path to>/lilypond/out/bin/lilypond "$@"

Save it as Lilypond (with a capital L to distinguish it from the stable lilypond), and make it executable:

chmod +x Lilypond

Then you can invoke the development version this way:

Lilypond foobar.ly

TODO: ADD

- other compilation tricks for developers

4.9 Build system

We currently use make and stepmake, which is complicated and only used by us. Hopefully this will change in the future.

Version-specific texinfo macros

• made with scripts/build/create-version-itexi.py and
  scripts/build/create-weblinks-itexi.py
• used extensively in the WEBSITE_ONLY_BUILD version of the website (made with ‘website.make’, used on lilypond.org)
• not (?) used in the main docs?
• the numbers in VERSION file: MINOR_VERSION should be 1 more than the last release, VERSION_DEVEL should be the last online release. Yes, VERSION_DEVEL is less than VERSION.

5. Documentation work

There are currently 11 manuals for LilyPond, not including the translations. Each book is available in HTML, PDF, and info. The documentation is written in a language called texinfo – this allows us to generate different output formats from a single set of source files.

To organize multiple authors working on the documentation, we use a Version Control System (VCS) called git, previously discussed in Starting with Git.

5.1 Introduction to documentation work

Our documentation tries to adhere to our Documentation policy. This policy contains a few items which may seem odd. One policy in particular is often questioned by potential contributors: we do not repeat material in the Notation Reference, and instead provide links to the “definitive” presentation of that information. Some people point out, with good reason, that this makes the documentation harder to read. If we repeated certain information in relevant places, readers would be less likely to miss that information.

That reasoning is sound, but we have two counter-arguments. First, the Notation Reference – one of five manuals for users to read – is already over 500 pages long. If we repeated material, we could easily exceed 1000 pages! Second, and much more importantly, LilyPond is an evolving project. New features are added, bugs are fixed, and bugs are discovered and documented. If features are discussed in multiple places, the documentation team must find every instance. Since the manual is so large, it is impossible for one person to have the location of every piece of information memorized, so any attempt to update the documentation will invariably omit a few places. This second concern is not at all theoretical; the documentation used to be plagued with inconsistent information.

If the documentation were targeted for a specific version – say, LilyPond 2.10.5 – and we had unlimited resources to spend on documentation, then we could avoid this second problem. But since LilyPond evolves (and that is a very good thing!), and since we have quite limited resources, this policy remains in place.

A few other policies (such as not permitting the use of tweaks in the main portion of NR 1+2) may also seem counter-intuitive, but they also stem from attempting to find the most effective use of limited documentation help.

Before undertaking any large documentation work, contributors are encouraged to contact the Documentation Meister.

5.2 Documentation suggestions

Small additions

For additions to the documentation,

1. Tell us where the addition should be placed. Please include both the section number and title (i.e. "LM 2.13 Printing lyrics").
2. Please write exact changes to the text.
3. A formal patch to the source code is not required; we can take care of the technical details.
4. Send the suggestions to the bug-lilypond mailing list as discussed in Contact.
5. Here is an example of a perfect documentation report:

   To: bug-lilypond@gnu.org
   From: helpful-user@example.net
   Subject: doc addition
   
   In LM 2.13 (printing lyrics), above the last line ("More options,
   like..."), please add:
   
   ----
   To add lyrics to a divided part, use blah blah blah.  For example,
   
   \score {
     \notes {blah <<blah>> }
     \lyrics {blah <<blah>> }
     blah blah blah
   }
   ----
   
   In addition, the second sentence of the first paragraph is
   confusing.  Please delete that sentence (it begins "Users
   often...") and replace it with this:
   ----
   To align lyrics with something, do this thing.
   ----
   
   Have a nice day,
   Helpful User
   

Larger contributions

To replace large sections of the documentation, the guidelines are stricter. We cannot remove parts of the current documentation unless we are certain that the new version is an improvement.

1. Ask on the lilypond-devel mailing list if such a rewrite is necessary; somebody else might already be working on this issue!
2. Split your work into small sections; this makes it much easier to compare the new and old documentation.
3. Please prepare a formal git patch.

Contributions that contain examples using overrides

Examples that use overrides, tweaks, customer Scheme functions etc. are (with very few exceptions) not included in the main text of the manuals; as there would be far too many, equally useful, candidates.

The correct way is to submit your example, with appropriate explanatory text and tags, to the LilyPond Snippet Repository (LSR). Snippets that have the “docs” tag can then be easily added as a selected snippet in the documentation. It will also appear automatically in the Snippets lists. See Introduction to LSR.

Snippets that don’t have the “docs” tage will still be searchable and viewable within the LSR, but will be not be included in the Snippets list or be able to be included as part of the main documentation.

Generally, any new snippets that have the “docs” tag are more carefully checked for syntax and formatting.

Announcing your snippet

Once you have followed these guidelines, please send a message to lilypond-devel with your documentation submissions. Unfortunately there is a strict ‘no top-posting’ check on the mailing list; to avoid this, add:

> I'm not top posting

(you must include the > ) to the top of your documentation addition.

We may edit your suggestion for spelling, grammar, or style, and we may not place the material exactly where you suggested, but if you give us some material to work with, we can improve the manual much faster.

Thanks for your interest!

5.3 Texinfo introduction and usage policy

5.3.1 Texinfo introduction

The language is called Texinfo; you can see its manual here:

http://www.gnu.org/software/texinfo/manual/texinfo/

However, you don’t need to read those docs. The most important thing to notice is that text is text. If you see a mistake in the text, you can fix it. If you want to change the order of something, you can cut-and-paste that stuff into a new location.

5.3.2 Documentation files

All manuals live in ‘Documentation/’.

In particular, there are four user manuals, their respective master source files are ‘learning.tely’ (LM, Learning Manual), ‘notation.tely’ (NR, Notation Reference), ‘music-glossary.tely’ (MG, Music Glossary), and ‘lilypond-program’ (AU). Each chapter is written in a separate file, ending in ‘.itely’ for files containing lilypond code, and ‘.itexi’ for files without lilypond code, located in a subdirectory associated to the manual (‘learning/’ for ‘learning.tely’, and so on); list the subdirectory of each manual to determine the filename of the specific chapter you wish to modify.

Developer manuals live in ‘Documentation/’ too. Currently there is only one: the Contributor’s Guide ‘contrib-guide.texi’ you are reading.

Snippet files are part of documentation, and the Snippet List (SL) lives in ‘Documentation/’ just like the manuals. For information about how to modify the snippet files and SL, see LSR work.

5.3.3 Sectioning commands

The Notation Reference uses section headings at four, occasionally five, levels.

• Level 1: @chapter
• Level 2: @section
• Level 3: @subsection
• Level 4: @unnumberedsubsubsec
• Level 5: @subsubsubheading

The first three levels are numbered in html, the last two are not. Numbered sections correspond to a single html page in the split html documents.

The first four levels always have accompanying nodes so they can be referenced and are also included in the ToC in html.

Most of the manual is written at level 4 under headings created with

@node Foo
@unnumberedsubsubsec Foo

Level 3 subsections are created with

@node Foo
@subsection Foo

Level 4 headings and menus must be preceded by level 3 headings and menus, and so on for level 3 and level 2. If this is not what is wanted, please use:

@subsubsubheading Foo

Please leave two blank lines above a @node; this makes it easier to find sections in texinfo.

Do not use any @ commands for a @node. They may be used for any @sub... sections or headings however.

not:
@node @code{Foo} Bar
@subsection @code{Foo} Bar

but instead:
@node Foo Bar
@subsection @code{Foo} Bar

No punctuation may be used in the node names. If the heading text uses punctuation (in particular, colons and commas) simply leave this out of the node name and menu.

@menu
* Foo Bar::
@end menu

@node Foo Bar
@subsection Foo: Bar

Backslashes must not be used in node names or section headings. If the heading text should include a backslash simply leave this out of the node name and menu and replace it with @bs{} in the heading text.

@menu
* The set command
@end menu

@node The set command
@subsection The @code{@bs{}set} command

References to such a node may use the third argument of the @ref command to display the texually correct heading.

@ref{The set command,,The @code{@bs{}set command}

With the exception of @ commands, \ commands and punctuation, the section name should match the node name exactly.

Sectioning commands (@node and @section) must not appear inside an @ignore. Separate those commands with a space, ie @n ode.

Nodes must be included inside a

@menu
* foo::
* bar::
@end menu

construct. These can be constructed with scripts: see Stripping whitespace and generating menus.

5.3.4 LilyPond formatting

• Most LilyPond examples throughout the documentation can be produced with:

  @lilypond[verbatim,quote,relative=1]
  

  or

  @lilypond[verbatim,quote,relative=2]
  

  If using any combination of \header{}, \score{} or \layout{} in your example, then you must omit the relative variable and either use absolute entry mode or an explicit \relative{} construction.

  If using \book{} in your example then you must also omit the relative variable and either use absolute entry mode or an explicit \relative{} construction. However, you must also include the papersize=X variable, where X is a defined paper size from within ‘scm/paper.scm’. This is to avoid the default a4 paper size being used and leaving too much unnecessary whitespace and potentially awkward page breaks in the PDFs.

  The preferred papersizes are a5, a6 or a8landscape.

  a8landscape works best for a single measure with a single title and/or single tagline:

  @lilypond[papersize=a8landscape,verbatim]
  \book {
    \header {
      title = "A scale in LilyPond"
    }
    \relative {
      c d e f
    }
  }
  @end lilypond
  

  and can also be used to easily show features that require page breaks (i.e. page numbers) without taking large amounts of space within the documentation. Do not use the quote option with this paper size.

  a5 or a6 paper sizes are best used for examples that have more than two measures of music or require multiple staves (i.e. to illustrate cross-staff features, RH and LH parts etc.) and where \book{} constructions are required or where a8landscape produces an example that is too cramped. Depending on the example the quote option may need to be omitted.

  In rare cases, other options may be used (or omitted), but ask first.

• Please avoid using extra spacing either after or within the @lilypond parameters.

  not:          @lilypond [verbatim, quote, relative=1]
  but instead:  @lilypond[verbatim,quote,relative=1]
  

• Inspirational headwords are produced with:

  @lilypondfile[quote,ragged-right,line-width=16\cm,staffsize=16]
  {pitches-headword.ly}
  

• LSR snippets are linked with:

  @lilypondfile[verbatim,quote,ragged-right,texidoc,doctitle]
  {filename.ly}
  

• Use two spaces for indentation in lilypond examples (no tabs).
• All engravers should have double-quotes around them:

  \consists "Spans_arpeggio_engraver"
  

  LilyPond does not strictly require this, but it is a useful convention to follow.

• All context or layout object strings should be prefaced with #. Again, LilyPond does not strictly require this, but it is helpful to get users accustomed to this scheme construct, i.e. \set Staff.instrumentName = #"cello"
• Try to avoid using #' or #` within when describing context or layout properties outside of an @example or @lilypond, unless the description explicitly requires it.

  i.e. “...setting the transparent property leaves the object where it is, but makes it invisible.”

• If possible, only write one bar per line.
• If you only have one bar per line, omit bar checks. If you must put more than one bar per line (not recommended), then include bar checks.
• Tweaks should, if possible, also occur on their own line.

  not:          \override TextScript #'padding = #3 c1^"hi"
  but instead:  \override TextScript #'padding = #3
                c1^"hi"
  

  excepted in Templates, where ‘doctitle’ may be omitted.

• Avoid long stretches of input code. Nobody is going to read them in print. Create small examples. However, this does not mean it has be minimal.
• Specify durations for at least the first note of every bar.
• If possible, end with a complete bar.
• Comments should go on their own line, and be placed before the line(s) to which they refer.
• For clarity, always use { } marks even if they are not technically required; i.e.

  not:
  
  \context Voice \repeat unfold 2 \relative c' {
    c2 d
  }
  
  but instead:
  
  \context Voice {
    \repeat unfold 2 {
      \relative c' {
        c2 d
      }
    }
  }
  

• Add a space around { } marks; i.e.

  not:          \chordmode{c e g}
  but instead:  \chordmode { c e g }
  

• Use { } marks for additional \markup format commands; i.e.

  not:          c^\markup \tiny\sharp
  but instead:  c^\markup { \tiny \sharp }
  

• Remove any space around < > marks; i.e.

  not:           < c e g > 4
  but instead:   <c e g>4
  

• Beam, slur and tie marks should begin immediately after the first note with beam and phrase marks ending immediately after the last.

  a8\( ais16[ b cis( d] b) cis4~ b' cis,\)
  

• If you want to work on an example outside of the manual (for easier/faster processing), use this header:

  \paper {
    indent = 0\mm
    line-width = 160\mm - 2.0 * 0.4\in
    line-width = #(- line-width (* mm  3.000000))
  }
  
  \layout {
  }
  

  You may not change any of these values. If you are making an example demonstrating special \paper{} values, contact the Documentation Editor.

5.3.5 Text formatting

• Lines should be less than 72 characters long. (We personally recommend writing with 66-char lines, but do not bother modifying existing material). Also see the recommendations for fixed-width fonts in the Syntax survey.
• Do not use tabs.
• Do not use spaces at the beginning of a line (except in @example or @verbatim environments), and do not use more than a single space between words. ‘makeinfo’ copies the input lines verbatim without removing those spaces.
• Use two spaces after a period.
• In examples of syntax, use @var{musicexpr} for a music expression.
• Don’t use @rinternals{} in the main text. If you’re tempted to do so, you’re probably getting too close to “talking through the code”. If you really want to refer to a context, use @code{} in the main text and @rinternals{} in the @seealso.

5.3.6 Syntax survey

Comments

• @c … — single line comment. ‘@c NOTE:’ is a comment which should remain in the final version. (gp only command ;)
• @ignore — multi-line comment:

  @ignore
  …
  @end ignore
  

Cross references

Enter the exact @node name of the target reference between the brackets (eg. ‘@ref{Syntax survey}’). Do not split a cross-reference across two lines – this causes the cross-reference to be rendered incorrectly in html documents.

• @ref{…} — link within current manual.
• @rchanges{…} — link to Changes.
• @rcontrib{…} — link to Contributor’s Guide.
• @ressay{…} — link to Engraving Essay.
• @rextend{…} — link to Extending LilyPond.
• @rglos{…} — link to the Music Glossary.
• @rinternals{…} — link to the Internals Reference.
• @rlearning{…} — link to Learning Manual.
• @rlsr{…} — link to a Snippet section.
• @rprogram{…} — link to Application Usage.
• @ruser{…} — link to Notation Reference.
• @rweb{…} — link to General Information.

External links

• @email{…} — create a mailto: E-mail link.
• @uref{URL[, link text]} — link to an external url. Use within an @example ... @end example.

  @example
  @uref{URL [, link text ]}
  @end example
  

Fixed-width font

• @code{…}, @samp{…} —

  Use the @code{…} command when referring to individual language-specific tokens (keywords, commands, engravers, scheme symbols, etc.) in the text. Ideally, a single @code{…} block should fit within one line in the PDF output.

  Use the @samp{…} command when you have a short example of user input, unless it constitutes an entire @item by itself, in which case @code{…} is preferable. Otherwise, both should only be used when part of a larger sentence within a paragraph or @item. Do not use @code{…} or @samp{…} inside an @example block, and do not use either as a free-standing paragraph; use @example instead.

  A single unindented line in the PDF has space for about 79 fixed-width characters (76 if indented). Within an @item there is space for about 75 fixed-width characters. Each additional level of @itemize or @enumerate shortens the line by about 4 columns.

  However, even short blocks of @code{…} and @samp{…} can run into the margin if the Texinfo line-breaking algorithm gets confused. Additionally, blocks that are longer than this may in fact print nicely; it all depends where the line breaks end up. If you compile the docs yourself, check the PDF output to make sure the line breaks are satisfactory.

  The Texinfo setting @allowcodebreaks is set to false in the manuals, so lines within @code{…} or @samp{…} blocks will only break at spaces, not at hyphens or underscores. If the block contains spaces, use @w{@code{…}} or @w{@samp{…}} to prevent unexpected line breaks.

  The Texinfo settings txicodequoteundirected and txicodequotebacktick are both set in the manuals, so backticks (`) and apostrophes (') placed within blocks of @code, @example, or @verbatim are not converted to left- and right-angled quotes (‘ ’) as they normally are within the text, so the apostrophes in ‘@w{@code{\relative c''}}’ will display correctly. However, these settings do not affect the PDF output for anything within a @samp block (even if it includes a nested @code block), so entering ‘@w{@samp{\relative c''}}’ wrongly produces ‘\relative c’’’ in PDF. Consequently, if you want to use a @samp{…} block which contains backticks or apostrophes, you should instead use ‘@q{@code{…}}’ (or ‘@q{@w{@code{…}}}’ if the block also contains spaces). Note that backslashes within @q{…} blocks must be entered as ‘@bs{}’, so the example above would be coded as ‘@q{@w{@code{@bs{}relative c''}}}’.

• @command{…} — Use when referring to command-line commands within the text (eg. ‘@command{convert-ly}’). Do not use inside an @example block.
• @example — Use for examples of program code. Do not add extraneous indentation (i.e. don’t start every line with whitespace). Use the following layout (notice the use of blank lines). Omit the @noindent if the text following the example starts a new paragraph:

  …text leading into the example…
  
  @example
  …
  @end example
  
  @noindent
  continuation of the text…
  

  Individual lines within an @example block should not exceed 74 characters; otherwise they will run into the margin in the PDF output, and may get clipped. If an @example block is part of an @item, individual lines in the @example block should not exceed 70 columns. Each additional level of @itemize or @enumerate shortens the line by about 4 columns.

  For long command line examples, if possible, use a trailing backslash to break up a single line, indenting the next line with 2 spaces. If this isn’t feasible, use ‘@smallexample … @end smallexample’ instead, which uses a smaller fontsize. Use @example whenever possible, but if needed, @smallexample can fit up to 90 characters per line before running into the PDF margin. Each additional level of @itemize or @enumerate shortens a @smallexample line by about 5 columns.

• @file{…} — Use when referring to filenames and directories in the text. Do not use inside an @example block.
• @option{…} — Use when referring to command-line options in the text (eg. ‘@option{--format}’). Do not use inside an @example block.
• @verbatim — Prints the block exactly as it appears in the source file (including whitespace, etc.). For program code examples, use @example instead. @verbatim uses the same format as @example.

  Individual lines within an @verbatim block should not exceed 74 characters; otherwise they will run into the margin in the PDF output, and may get clipped. If an @verbatim block is part of an @item, individual lines in the @verbatim block should not exceed 70 columns. Each additional level of @itemize or @enumerate shortens the line by about 4 columns.

Indexing

• @cindex … — General index. Please add as many as you can. Don’t capitalize the first word.
• @funindex … — is for a \lilycommand.

Lists

• @enumerate — Create an ordered list (with numbers). Always put ‘@item’ on its own line. As an exception, if all the items in the list are short enough to fit on single lines, placing them on the ‘@item’ lines is also permissible. ‘@item’ and ‘@end enumerate’ should always be preceded by a blank line.

  @enumerate
  
  @item
  A long multi-line item like this one must begin
  on a line of its own and all the other items in
  the list must do so too.
  
  @item
  Even short ones
  
  @end enumerate
  

  @enumerate
  
  @item Short item
  
  @item Short item
  
  @end enumerate
  

• @itemize — Create an unordered list (with bullets). Use the same format as @enumerate. Do not use ‘@itemize @bullet’.

Special characters

• --, --- — Create an en dash (–) or an em dash (—) in the text. To print two or three literal hyphens in a row, wrap one of them in a @w{…} (eg. ‘-@w{-}-’).
• @@, @{, @} — Create an at-sign (@), a left curly bracket ({), or a right curly bracket (}).
• @bs{} — Create a backslash within a @q{…}, @qq{…}, or @warning{…} block. This is a custom LilyPond macro, not a builtin @-command in Texinfo. Texinfo would also allow ‘\\’, but this breaks the PDF output.
• @tie{} — Create a variable-width non-breaking space in the text (use ‘@w{ }’ for a single fixed-width non-breaking space). Variables or numbers which consist of a single character (probably followed by a punctuation mark) should be tied properly, either to the previous or the next word. Example: ‘The letter@tie{}@q{I} is skipped’

Miscellany

• @notation{…} — refers to pieces of notation, e.g. ‘@notation{clef}’. Also use for specific lyrics (‘the @notation{A - men} is centered’). Only use once per subsection per term.
• @q{…} — Single quotes. Used for ‘vague’ terms. To get a backslash (\), you must use ‘@bs{}’.
• @qq{…} — Double quotes. Used for actual quotes (“he said”) or for introducing special input modes. To get a backslash (\), you must use ‘@bs{}’.
• @var{…} — Use for metasyntactic variables (such as foo, bar, arg1, etc.). In most cases, when the @var{…} command appears in the text (and not in an @example block) it should be wrapped with an appropriate texinfo code-highlighting command (such as @code, @samp, @file, @command, etc.). For example: ‘@code{@var{foo}}’, ‘@file{@var{myfile.ly}}’, ‘@samp{git checkout @var{branch}}’, etc. This improves readability in the PDF and HTML output.
• @version{} — Return the current LilyPond version string. Use ‘@w{@version{}}’ if it’s at the end of a line (to prevent an ugly line break in PDF); use ‘@w{"@version{}"}’ if you need it in quotes.
• @w{…} — Do not allow any line breaks.
• @warning{…} — produces a “Note: ” box. Use for important messages. To get a backslash (\), you must use ‘@bs{}’.

5.3.7 Other text concerns

• References must occur at the end of a sentence, for more information see the texinfo manual. Ideally this should also be the final sentence of a paragraph, but this is not required. Any link in a doc section must be duplicated in the @seealso section at the bottom.
• Introducing examples must be done with

  . (i.e. finish the previous sentence/paragraph)
  : (i.e. `in this example:')
  , (i.e. `may add foo with the blah construct,')
  

  The old “sentence runs directly into the example” method is not allowed any more.

• Abbrevs in caps, e.g., HTML, DVI, MIDI, etc.
• Colon usage
  1. To introduce lists
  2. When beginning a quote: “So, he said,...”.

     This usage is rarer. Americans often just use a comma.

  3. When adding a defining example at the end of a sentence.
• Non-ASCII characters which are in utf-8 should be directly used; this is, don’t say ‘Ba@ss{}tuba’ but ‘Baßtuba’. This ensures that all such characters appear in all output formats.

5.4 Documentation policy

5.4.1 Books

There are four parts to the documentation: the Learning Manual, the Notation Reference, the Program Reference, and the Music Glossary.

• Learning Manual:

  The LM is written in a tutorial style which introduces the most important concepts, structure and syntax of the elements of a LilyPond score in a carefully graded sequence of steps. Explanations of all musical concepts used in the Manual can be found in the Music Glossary, and readers are assumed to have no prior knowledge of LilyPond. The objective is to take readers to a level where the Notation Reference can be understood and employed to both adapt the templates in the Appendix to their needs and to begin to construct their own scores. Commonly used tweaks are introduced and explained. Examples are provided throughout which, while being focussed on the topic being introduced, are long enough to seem real in order to retain the readers’ interest. Each example builds on the previous material, and comments are used liberally. Every new aspect is thoroughly explained before it is used.

  Users are encouraged to read the complete Learning Manual from start-to-finish.

• Notation Reference: a (hopefully complete) description of LilyPond input notation. Some material from here may be duplicated in the Learning Manual (for teaching), but consider the NR to be the "definitive" description of each notation element, with the LM being an "extra". The goal is _not_ to provide a step-by-step learning environment – do not avoid using notation that has not be introduced previously in the NR (for example, use \break if appropriate). This section is written in formal technical writing style.

  Avoid duplication. Although users are not expected to read this manual from start to finish, they should be familiar with the material in the Learning Manual (particularly “Fundamental Concepts”), so do not repeat that material in each section of this book. Also watch out for common constructs, like ^ - _ for directions – those are explained in NR 3. In NR 1, you can write: DYNAMICS may be manually placed above or below the staff, see @ref{Controlling direction and placement}.

  Most tweaks should be added to LSR and not placed directly in the ‘.itely’ file. In some cases, tweaks may be placed in the main text, but ask about this first.

  Finally, you should assume that users know what the notation means; explaining musical concepts happens in the Music Glossary.

• Application Usage: information about using the program lilypond with other programs (lilypond-book, operating systems, GUIs, convert-ly, etc). This section is written in formal technical writing style.

  Users are not expected to read this manual from start to finish.

• Music Glossary: information about the music notation itself. Explanations and translations about notation terms go here.

  Users are not expected to read this manual from start to finish.

• Internals Reference: not really a documentation book, since it is automagically generated from the source, but this is its name.

5.4.2 Section organization

• The order of headings inside documentation sections should be:

  main docs
  @predefined
  @endpredefined
  @snippets
  @seealso
  @knownissues
  

• You must include a @seealso.
  • The order of items inside the @seealso section is

    Music Glossary:
    @rglos{foo},
    @rglos{bar}.
    
    Learning Manual:
    @rlearning{baz},
    @rlearning{foozle}.
    
    Notation Reference:
    @ruser{faazle},
    @ruser{boo}.
    
    Application Usage:
    @rprogram{blah}.
    
    Essay on automated music engraving:
    @ressay{yadda}.
    
    Extending LilyPond:
    @rextend{frob}.
    
    Installed Files:
    @file{path/to/dir/blahz}.
    
    Snippets: @rlsr{section}.
    
    Internals Reference:
    @rinternals{fazzle},
    @rinternals{booar}.
    

  • If there are multiple entries, separate them by commas but do not include an ‘and’.
  • Always end with a period.
  • Place each link on a new line as above; this makes it much easier to add or remove links. In the output, they appear on a single line.

    ("Snippets" is REQUIRED; the others are optional)

  • Any new concepts or links which require an explanation should go as a full sentence(s) in the main text.
  • Don’t insert an empty line between @seealso and the first entry! Otherwise there is excessive vertical space in the PDF output.
• To create links, use @ref{} if the link is within the same manual.
• @predefined ... @endpredefined is for commands in ‘ly/*-init.ly’
• Do not include any real info in second-level sections (i.e. 1.1 Pitches). A first-level section may have introductory material, but other than that all material goes into third-level sections (i.e. 1.1.1 Writing Pitches).
• The @knownissues should not discuss any issues that are in the tracker, unless the issue is Priority-Postponed. The goal is to discuss any overall architecture or syntax decisions which may be interpreted as bugs. Normal bugs should not be discussed here, because we have so many bugs that it would be a huge task to keep the @knownissues current and accurate all the time.

5.4.3 Checking cross-references

Cross-references between different manuals are heavily used in the documentation, but they are not checked during compilation. However, if you compile the documentation, a script called check_texi_refs can help you with checking and fixing these cross-references; for information on usage, cd into a source tree where documentation has been built, cd into Documentation and run:

make check-xrefs
make fix-xrefs

Note that you have to find yourself the source files to fix cross-references in the generated documentation such as the Internals Reference; e.g. you can grep scm/ and lily/.

Also of interest may be the linkdoc checks on kainhofer.com. Be warned that these docs are not completely rebuilt every day, so it might not accurately reflect the current state of the docs.

http://kainhofer.com/~lilypond/linkdoc/

5.4.4 General writing

• Do not forget to create @cindex entries for new sections of text. Enter commands with @funindex, i.e.

  @cindex pitches, writing in different octaves
  @funindex \relative
  

  Do not bother with the @code{} (they are added automatically). These items are added to both the command index and the unified index. Both index commands should go in front of the actual material.

• @cindex entries should not be capitalized, i.e.

  @cindex time signature
  

  is preferred instead of “Time signature”. Only use capital letters for musical terms which demand them, e.g. “D.S. al Fine”.

• For scheme function index entries, only include the final part, i.e.

  @funindex modern-voice-cautionary
       and NOT
  @funindex #(set-accidental-style modern-voice-cautionary)
  

• Use American spelling. LilyPond’s internal property names use this convention.
• Here is a list of preferred terms to be used:
  • Simultaneous NOT concurrent.
  • Measure: the unit of music.
  • Bar line: the symbol delimiting a measure NOT barline.
  • Note head NOT notehead.
  • Chord construct NOT just chord (when referring to < ... >)
  • Staff NOT stave.
  • Staves NOT Staffs: Phrases such as ‘multiple @internalsref{Staff}s’ should be rephrased to ‘multiple @internalsref{Staff} contexts’.

5.4.5 Technical writing style

These refer to the NR. The LM uses a more gentle, colloquial style.

• Do not refer to LilyPond in the text. The reader knows what the manual is about. If you do, capitalization is LilyPond.
• If you explicitly refer to ‘lilypond’ the program (or any other command to be executed), write @command{lilypond}.
• Do not explicitly refer to the reader/user. There is no one else besides the reader and the writer.
• Avoid contractions (don’t, won’t, etc.). Spell the words out completely.
• Avoid abbreviations, except for commonly used abbreviations of foreign language terms such as etc. and i.e.
• Avoid fluff (“Notice that,” “as you can see,” “Currently,”).
• The use of the word ‘illegal’ is inappropriate in most cases. Say ‘invalid’ instead.

5.5 Tips for writing docs

In the NR, I highly recommend focusing on one subsection at a time. For each subsection,

• check the mundane formatting. Are the headings (@predefined, @seealso, etc.) in the right order?
• add any appropriate index entries.
• check the links in the @seealso section – links to music glossary, internal references, and other NR sections are the main concern. Check for potential additions.
• move LSR-worthy material into LSR. Add the snippet, delete the material from the ‘.itely’ file, and add a @lilypondfile command.
• check the examples and descriptions. Do they still work? Do not assume that the existing text is accurate/complete; some of the manual is highly out of date.
• is the material in the @knownissues still accurate?
• can the examples be improved (made more explanatory), or is there any missing info? (feel free to ask specific questions on -user; a couple of people claimed to be interesting in being “consultants” who would help with such questions)

In general, I favor short text explanations with good examples – “an example is worth a thousand words”. When I worked on the docs, I spent about half my time just working on those tiny lilypond examples. Making easily-understandable examples is much harder than it looks.

Tweaks

In general, any \set or \override commands should go in the “select snippets” section, which means that they should go in LSR and not the ‘.itely’ file. For some cases, the command obviously belongs in the “main text” (i.e. not inside @predefined or @seealso or whatever) – instrument names are a good example of this.

\set Staff.instrumentName = #"foo"

On the other side of this,

\override Score.Hairpin #'after-line-breaking = ##t

clearly belongs in LSR.

I’m quite willing to discuss specific cases if you think that a tweaks needs to be in the main text. But items that can go into LSR are easier to maintain, so I’d like to move as much as possible into there.

It would be “nice” if you spent a lot of time crafting nice tweaks for users… but my recommendation is not to do this. There’s a lot of doc work to do without adding examples of tweaks. Tweak examples can easily be added by normal users by adding them to the LSR.

One place where a documentation writer can profitably spend time writing or upgrading tweaks is creating tweaks to deal with known issues. It would be ideal if every significant known issue had a workaround to avoid the difficulty.

See also

Adding and editing snippets.

5.6 Scripts to ease doc work

5.6.1 Scripts to test the documentation

Building only one section of the documentation

In order to save build time, a script is available to build only one section of the documentation in English with a default html appearance.

You can build a section of the documentation with:

scripts/auxiliar/doc-section.sh MANUAL SECTION

where SECTION is the name of the file containing the section to be built, and MANUAL is replaced by the name of the directory containing the section. So, for example, to build section 1.1 of the Notation Reference, use the command:

scripts/auxiliar/doc-section.sh notation pitches

You can then see the generated document for the section at

tempdocs/pitches/out/pitches.html

According to Lilypond issue 1236, the location of the lilypond git tree is taken from $LILYPOND_GIT if specified, otherwise it is auto-detected.

It is assumed that compilation takes place in the ‘build/’ subdirectory, but this can be overridden by setting the environment variable LILYPOND_BUILD_DIR.

Similarly, output defaults to ‘build/tempdocs/’ but this can be overridden by setting the environment variable LILYPOND_TEMPDOCS.

This script will not work for building sections of the Contributors’ guide. For building sections of the Contributors’ Guide, use:

scripts/auxiliar/cg-section.sh SECTION

where SECTION is the name of the file containing the sections to be built. For example, to build section 4 of the Contributors’ guide, use:

scripts/auxiliar/cg-section.sh doc-work

cg-section.sh uses the same environment variables and corresponding default values as doc-section.sh.

5.6.2 Scripts to create documentation

Stripping whitespace and generating menus

To automatically regenerate @menu portions and strip whitespace, use:

scripts/auxiliar/node-menuify.py FILENAME

If you are adding documentation that requires new menus, you will need to add a blank @menu section:

@menu
@end menu

Stripping whitespace only

To remove extra whitespace from the ends of lines, run

scripts/auxiliar/strip-whitespace.py FILENAME

Updating doc with convert-ly

Don’t. This should be done by programmers when they add new features. If you notice that it hasn’t been done, complain to lilypond-devel.

5.7 Docstrings in scheme

Material in the Internals reference is generated automatically from our source code. Any doc work on Internals therefore requires modifying files in ‘scm/*.scm’. Texinfo is allowed in these docstrings.

Most documentation writers never touch these, though. If you want to work on them, please ask for help.

5.8 Translating the documentation

The mailing list translations@lilynet.net is dedicated to LilyPond web site and documentation translation; on this list, you will get support from the Translations Meister and experienced translators, and we regularly discuss translation issues common to all languages. All people interested in LilyPond translations are invited to subscribe to this list regardless of the amount of their contribution, by sending an email to translations-request@lilynet.net with subject subscribe and an empty message body. Unless mentioned explicitly, or except if a translations coordinator contacts you privately, you should send questions, remarks and patches to the list translations@lilynet.net. Please note that traffic is high on the English-speaking list lilypond-user@gnu.org, so it may take some time before your request or contribution is handled.

5.8.1 Getting started with documentation translation

First, get the sources of branch translation from the Git repository, see Starting with Git.

Translation requirements

Working on LilyPond documentation translations requires the following pieces of software, in order to make use of dedicated helper tools:

• Python 2.4 or higher,
• GNU Make,
• Gettext,
• Git.

It is not required to build LilyPond and the documentation to translate the documentation. However, if you have enough time and motivation and a suitable system, it can be very useful to build at least the documentation so that you can check the output yourself and more quickly; if you are interested, see Compiling.

Before undertaking any large translation work, contributors are encouraged to contact the Translation Meister.

Which documentation can be translated

The makefiles and scripts infrastructure currently supports translation of the following documentation:

• the web site, the Learning Manual, the Notation Reference and Application Usage – Texinfo source, PDF and HTML output; Info output might be added if there is enough demand for it;
• the Changes document.

Support for translating the following pieces of documentation should be added soon, by decreasing order of priority:

• automatically generated documentation: markup commands, predefined music functions;
• the Snippets List;
• the Internals Reference.

Starting translation in a new language

At top of the source directory, do

./autogen.sh

or (if you want to install your self-compiled LilyPond locally)

./autogen.sh --prefix=$HOME

If you want to compile LilyPond – which is almost required to build the documentation, but is not required to do translation only – fix all dependencies and rerun ./configure (with the same options as for autogen.sh).

Then cd into ‘Documentation/’ and run

make ISOLANG=MY-LANGUAGE new-lang

where MY-LANGUAGE is the ISO 639 language code.

Finally, add a language definition for your language in ‘python/langdefs.py’.

5.8.2 Documentation translation details

Please follow all the instructions with care to ensure quality work.

All files should be encoded in UTF-8.

Files to be translated

Translation of ‘Documentation/foo/bar’ should be ‘Documentation/LANG/foo/bar’. Unmentioned files should not be translated.

Priorities:

• 1. delivery,
• 2. 3. 4. 5. 6. later,
• 7. optional.

Files of priority 1 should be submitted along all files generated by starting a new language in the same commit and thus a unique patch, and the translation of files marked with priority 2 should be committed to Git at the same time and thus sent in a single patch. Files marked with priority 3 or more may be submitted individually. Word counts (excluding LilyPond snippets) are given for each file. For knowing how to commit your work to Git, then make patches of your new translations as well as corrections and updates, see Basic Git procedures.

-1- Web site
616   web.texi
4937  web/introduction.itexi
1201  web/download.itexi
1139  macros.itexi
7977  po/lilypond-doc.pot (translate to po/MY_LANGUAGE.po)
0     search-box.ihtml
---   lilypond-texi2html.init (section TRANSLATIONS)
15870 total

-2- Tutorial
1284  web/manuals.itexi
124   learning.tely
2578  learning/tutorial.itely
4396  learning/common-notation.itely
8382  total

-3- Fundamental Concepts, starting of Usage and Community
11144 learning/fundamental.itely -- Fundamental concepts
135   usage.tely
4537  usage/running.itely
1484  usage/updating.itely
3073  web/community.itexi
20373 total

-4- Rest of Learning manual and Suggestions on writing LilyPond files
16191 learning/tweaks.itely -- Tweaking output
372   learning/templates.itely -- Templates
2692  usage/suggestions.itely -- Suggestions on writing LilyPond files
19255 total

-5- Notation reference
326   notation.tely
91    notation/notation.itely -- Musical notation
4990  notation/pitches.itely
6890  notation/rhythms.itely
1793  notation/expressive.itely
1050  notation/repeats.itely
2821  notation/simultaneous.itely
2476  notation/staff.itely
954   notation/editorial.itely
2816  notation/text.itely
81    notation/specialist.itely -- Specialist notation
5190  notation/vocal.itely
1972  notation/chords.itely
702   notation/piano.itely
811   notation/percussion.itely
826   notation/guitar.itely
66    notation/strings.itely
242   notation/bagpipes.itely
5375  notation/ancient.itely
10392 notation/input.itely -- Input syntax
2164  notation/non-music.itely -- Non-musical notation
12256 notation/spacing.itely -- Spacing issues
15289 notation/changing-defaults.itely -- Changing defaults
5187  notation/programming-interface.itely -- Interfaces for programmers
2176  notation/notation-appendices.itely -- Notation manual tables
252   notation/cheatsheet.itely -- Cheat sheet
87188 total

-6- Rest of Application Usage
4137  usage/lilypond-book.itely -- LilyPond-book
1122  usage/converters.itely -- Converting from other formats
5259  total

-7- Appendices whose translation is optional
326   essay/literature.itely
1222  learning/scheme-tutorial.itely (should be revised first)
1548  total

In addition, not listed above, Snippets’ titles and descriptions should be translated; they are a part of the Notation Reference and therefore their priority is 5.

Translating the Web site and other Texinfo documentation

Every piece of text should be translated in the source file, except Texinfo comments, text in @lilypond blocks and a few cases mentioned below.

Node names are translated, but the original node name in English should be kept as the argument of @translationof put after the section title; that is, every piece in the original file like

@node Foo bar
@section_command Bar baz

should be translated as

@node translation of Foo bar
@section_command translation of Bar baz
@translationof Foo bar

The argument of @rglos commands and the first argument of @rglosnamed commands must not be translated, as it is the node name of an entry in Music Glossary.

Every time you translate a node name in a cross-reference, i.e. the argument of commands @ref, @rprogram, @rlearning, @rlsr, @ruser or the first argument of their *named variants, you should make sure the target node is defined in the correct source file; if you do not intend to translate the target node right now, you should at least write the node definition (that is, the @node @section_commmand @translationof trio mentioned above) in the expected source file and define all its parent nodes; for each node you have defined this way but have not translated, insert a line that contains @untranslated. That is, you should end up for each untranslated node with something like

@node translation of Foo bar
@section_command translation of Bar baz
@translationof Foo bar

@untranslated

Please think of the fact that it may not make sense translating everything in some Texinfo files, and you may take distance from the original text; for instance, in the translation of the web site section Community, you may take this into account depending on what you know the community in your language is willing to support, which is possible only if you personally assume this support, or there exists a public forum or mailing list listed in Community for LilyPond in your language:

• Bug reports: this page should be translated only if you know that every bug report sent on your language’s mailing list or forum will be handled by someone who will translate it to English and send it on bug-lilypond or add an issue in the tracker, then translate back the reply from developers.
• Help us: this page should be translated very freely, and possibly not at all: ask help for contributing to LilyPond for tasks that LilyPond community in your language is able and going to handle.

In any case, please mark in your work the sections which do not result from the direct translation of a piece of English translation, using comments i.e. lines starting with ‘@c’.

Finally, press in Emacs <C-c C-u C-a> to update or generate menus. This process should be made easier in the future, when the helper script texi-langutils.py and the makefile target are updated.

Some pieces of text manipulated by build scripts that appear in the output are translated in a ‘.po’ file – just like LilyPond output messages – in ‘Documentation/po’. The Gettext domain is named lilypond-doc, and unlike lilypond domain it is not managed through the Free Translation Project.

Take care of using typographic rules for your language, especially in ‘macros.itexi’.

If you wonder whether a word, phrase or larger piece of text should be translated, whether it is an argument of a Texinfo command or a small piece sandwiched between two Texinfo commands, try to track whether and where it appears in PDF and/or HTML output as visible text. This piece of advice is especially useful for translating ‘macros.itexi’.

Please keep verbatim copies of music snippets (in @lilypond blocs). However, some music snippets containing text that shows in the rendered music, and sometimes translating this text really helps the user to understand the documentation; in this case, and only in this case, you may as an exception translate text in the music snippet, and then you must add a line immediately before the @lilypond block, starting with

@c KEEP LY

Otherwise the music snippet would be reset to the same content as the English version at next make snippet-update run – see Updating documentation translation.

When you encounter

@lilypondfile[<number of fragment options>,texidoc]{filename.ly}

in the source, open ‘Documentation/snippets/filename.ly’, translate the texidoc header field it contains, enclose it with texidocMY-LANGUAGE = " and ", and write it into ‘Documentation/MY-LANGUAGE/texidocs/filename.texidoc’. Additionally, you may translate the snippet’s title in doctitle header field, in case doctitle is a fragment option used in @lilypondfile; you can do this exactly the same way as texidoc. For instance, ‘Documentation/MY-LANGUAGE/texidocs/filename.texidoc’ may contain

doctitlees = "Spanish title baz"
texidoces = "
Spanish translation blah
"

@example blocks need not be verbatim copies, e.g. variable names, file names and comments should be translated.

Finally, please carefully apply every rule exposed in Texinfo introduction and usage policy, and Documentation policy. If one of these rules conflicts with a rule specific to your language, please ask the Translation meister on translations@lilynet.net list and/or the Documentation Editors on lilypond-devel@gnu.org list.

Adding a Texinfo manual

In order to start translating a new manual whose basename is FOO, do

cd Documentation/MY-LANGUAGE
cp ../FOO.tely .
mkdir FOO
cp web/GNUmakefile FOO

then append FOO to variable SUBDIRS in Documentation/MY-LANGUAGE/GNUmakefile, then translate file MY-LANGUAGE/FOO.tely and run skeleton-update:

cd Documentation/
make ISOLANG=MY-LANGUAGE TEXI_LANGUTIL_FLAGS=--head-only skeleton-update

Your are now ready to translate the new manual exactly like the web site or the Learning Manual.

5.8.3 Documentation translation maintenance

Several tools have been developed to make translations maintenance easier. These helper scripts make use of the power of Git, the version control system used for LilyPond development.

You should use them whenever you would like to update the translation in your language, which you may do at the frequency that fits your and your cotranslators’ respective available times. In the case your translation is up-do-date (which you can discover in the first subsection below), it is enough to check its state every one or two weeks. If you feel overwhelmed by the quantity of documentation to be updated, see Maintaining without updating translations.

Check state of translation

First pull from Git – see Pulling and rebasing, but DO NOT rebase unless you are sure to master the translation state checking and updating system – then cd into ‘Documentation/’ (or at top of the source tree, replace make with make -C Documentation) and run

make ISOLANG=MY_LANGUAGE check-translation

This presents a diff of the original files since the most recent revision of the translation. To check a single file, cd into ‘Documentation/’ and run

make CHECKED_FILES=MY_LANGUAGE/manual/foo.itely check-translation

In case this file has been renamed since you last updated the translation, you should specify both old and new file names, e.g. CHECKED_FILES=MY_LANGUAGE/{manual,user}/foo.itely.

To see only which files need to be updated, do

make ISOLANG=MY_LANGUAGE check-translation | grep 'diff --git'

To avoid printing terminal colors control characters, which is often desirable when you redirect output to a file, run

make ISOLANG=MY_LANGUAGE NO_COLOR=1 check-translation

You can see the diffs generated by the commands above as changes that you should make in your language to the existing translation, in order to make your translation up to date.

Global state of the translation is recorded in ‘Documentation/translations.itexi’, which is used to generate Translations status page. To update that page, do from ‘Documentation/’

make translation-status

This will also leave ‘out/translations-status.txt’, which contains up-to-dateness percentages for each translated file, and update word counts of documentation files in this Guide.

See also

Maintaining without updating translations.

Updating documentation translation

Instead of running check-translation, you may want to run update-translation, which will run your favorite text editor to update files. First, make sure environment variable EDITOR is set to a text editor command, then run from ‘Documentation/’

make ISOLANG=MY_LANGUAGE update-translation

or to update a single file

make CHECKED_FILES=MY_LANGUAGE/manual/foo.itely update-translation

For each file to be updated, update-translation will open your text editor with this file and a diff of the file in English; if the diff cannot be generated or is bigger than the file in English itself, the full file in English will be opened instead.

Texinfo skeleton files, i.e. ‘.itely’ files not yet translated, containing only the first node of the original file in English can be updated automatically: whenever make check-translation shows that such files should be updated, run from ‘Documentation/’

make ISOLANG=MY_LANGUAGE skeleton-update

‘.po’ message catalogs in ‘Documentation/po/’ may be updated by issuing from ‘Documentation/’ or ‘Documentation/po/’

make po-update

Updating music snippets can quickly become cumbersome, as most snippets should be identical in all languages. Fortunately, there is a script that can do this odd job for you (run from ‘Documentation/’):

make ISOLANG=MY_LANGUAGE snippet-update

This script overwrites music snippets in ‘MY_LANGUAGE/foo/every.itely’ with music snippets from ‘foo/every.itely’. It ignores skeleton files, and keeps intact music snippets preceded with a line starting with @c KEEP LY; it reports an error for each ‘.itely’ that has not the same music snippet count in both languages. Always use this script with a lot of care, i.e. run it on a clean Git working tree, and check the changes it made with git diff before committing; if you don’t do so, some @lilypond snippets might be broken or make no sense in their context.

Finally, a command runs the three update processes above for all enabled languages (from ‘Documentation/’):

make all-translations-update

Use this command with caution, and keep in mind it will not be really useful until translations are stabilized after the end of GDP and GOP.

See also

Maintaining without updating translations, Adding and editing snippets.

Updating translation committishes

At the beginning of each translated file except PO files, there is a committish which represents the revision of the sources which you have used to translate this file from the file in English.

When you have pulled and updated a translation, it is very important to update this committish in the files you have completely updated (and only these); to do this, first commit possible changes to any documentation in English which you are sure to have done in your translation as well, then replace in the up-to-date translated files the old committish by the committish of latest commit, which can be obtained by doing

git rev-list HEAD |head -1

Most of the changes in the LSR snippets included in the documentation concern the syntax, not the description inside texidoc="". This implies that quite often you will have to update only the committish of the matching .texidoc file. This can be a tedious work if there are many snippets to be marked as up do date. You can use the following command to update the committishes at once:

cd Documentation/LANG/texidocs
sed -i -r 's/[0-9a-z]{40}/NEW-COMMITTISH/' *.texidoc

See also

LSR work.

5.8.4 Translations management policies

These policies show the general intent of how the translations should be managed, they aim at helping translators, developers and coordinators work efficiently.

Maintaining without updating translations

Keeping translations up to date under heavy changes in the documentation in English may be almost impossible, especially as during the former Grand Documentation Project (GDP) or the Grand Organization Project (GOP) when a lot of contributors brings changes. In addition, translators may be — and that is a very good thing — involved in these projects too.

it is possible — and even recommended — to perform some maintenance that keeps translated documentation usable and eases future translation updating. The rationale below the tasks list motivates this plan.

The following tasks are listed in decreasing priority order.

1. Update macros.itexi. For each obsolete macro definition, if it is possible to update macro usage in documentation with an automatic text or regexp substitution, do it and delete the macro definition from ‘macros.itexi’; otherwise, mark this macro definition as obsolete with a comment, and keep it in ‘macros.itexi’ until the documentation translation has been updated and no longer uses this macro.
2. Update ‘*.tely’ files completely with make check-translation – you may want to redirect output to a file because of overwhelming output, or call check-translation.py on individual files, see Check state of translation.
3. In ‘.itelys’, match sections and .itely file names with those from English docs, which possibly involves moving nodes contents in block between files, without updating contents itself. In other words, the game is catching where has gone each section. In Learning manual, and in Notation Reference sections which have been revised in GDP, there may be completely new sections: in this case, copy @node and @section-command from English docs, and add the marker for untranslated status @untranslated on a single line. Note that it is not possible to exactly match subsections or subsubsections of documentation in English, when contents has been deeply revised; in this case, keep obsolete (sub)subsections in the translation, marking them with a line @c obsolete just before the node.

   Emacs with Texinfo mode makes this step easier:

   • without Emacs AucTeX installed, <C-c C-s> shows structure of current Texinfo file in a new buffer *Occur*; to show structure of two files simultaneously, first split Emacs window in 4 tiles (with <C-x 1> and <C-x 2>), press <C-c C-s> to show structure of one file (e.g. the translated file), copy *Occur* contents into *Scratch*, then press <C-c C-s> for the other file.

     If you happen to have installed AucTeX, you can either call the macro by doing <M-x texinfo-show-structure> or create a key binding in your ‘~/.emacs’, by adding the four following lines:

     (add-hook 'Texinfo-mode-hook
               '(lambda ()
                  (define-key Texinfo-mode-map "\C-cs"
                   'texinfo-show-structure)))
     

     and then obtain the structure in the *Occur* buffer with <C-c s>.

   • Do not bother updating @menus when all menu entries are in the same file, just do <C-c C-u C-a> (“update all menus”) when you have updated all the rest of the file.
   • Moving to next or previous node using incremental search: press <C-s> and type node (or <C-s @node> if the text contains the word ‘node’) then press <C-s> to move to next node or <C-r> to move to previous node. Similar operation can be used to move to the next/previous section. Note that every cursor move exits incremental search, and hitting <C-s> twice starts incremental search with the text entered in previous incremental search.
   • Moving a whole node (or even a sequence of nodes): jump to beginning of the node (quit incremental search by pressing an arrow), press <C-SPACE>, press <C-s node> and repeat <C-s> until you have selected enough text, cut it with <C-w> or <C-x>, jump to the right place (moving between nodes with the previous hint is often useful) and paste with <C-y> or <C-v>.
4. Update sections finished in the English documentation; check sections status at

   http://lilypondwiki.tuxfamily.org/index.php?title=Documentation_coordination.

5. Update documentation PO. It is recommended not to update strings which come from documentation that is currently deeply revised in English, to avoid doing the work more than once.
6. Fix broken cross-references by running (from ‘Documentation/’)

   make ISOLANG=YOUR-LANGUAGE fix-xrefs
   

   This step requires a successful documentation build (with make doc). Some cross-references are broken because they point to a node that exists in the documentation in English, which has not been added to the translation; in this case, do not fix the cross-reference but keep it "broken", so that the resulting HTML link will point to an existing page of documentation in English.

Rationale

You may wonder if it would not be better to leave translations as-is until you can really start updating translations. There are several reasons to do these maintenance tasks right now.

• This will have to be done sooner or later anyway, before updating translation of documentation contents, and this can already be done without needing to be redone later, as sections of documentation in English are mostly revised once. However, note that not all documentation sectioning has been revised in one go, so all this maintenance plan has to be repeated whenever a big reorganization is made.
• This just makes translated documentation take advantage of the new organization, which is better than the old one.
• Moving and renaming sections to match sectioning of documentation in English simplify future updating work: it allows updating the translation by side-by-side comparison, without bothering whether cross-reference names already exist in the translation.
• Each maintenance task except ‘Updating PO files’ can be done by the same person for all languages, which saves overall time spent by translators to achieve this task: the node names and section titles are in English, so you can do. It is important to take advantage of this now, as it will be more complicated (but still possible) to do step 3 in all languages when documentation is compiled with texi2html and node names are directly translated in source files.

Managing documentation translation with Git

This policy explains how to manage Git branches and commit translations to Git.

• Translation work is made on translation branch. This branch is merged on staging once a week, approximately. Then, master branch is merged on translation, where the check-translation script (see Check state of translation) shows changes in English docs which should be translated, and the cycle starts again.
• Translations may be pushed directly to staging only if they do not break compilation of LilyPond and its documentation. Those changes could be pushed to translation too, or alternatively translators could wait until they come from master the next time it is merged on translation. Similarly, changes matching stable/X.Y are preferably made on X.Ytranslation.
• translation Git branch may be merged into staging branch only if LilyPond (make all) and documentation (make doc) compile successfully.
• make and make doc are usually successful in master Git branch because those tests should have already succeeded in staging branch before merging. master branch may be merged into translation when significant changes had been made in documentation in English in master branch.
• General maintenance may be done by anybody who knows what he does in documentation in all languages, without informing translators first. General maintenance include simple text substitutions (e.g. automated by sed), compilation fixes, updating Texinfo or lilypond-book commands, updating macros, updating ly code, fixing cross-references, and operations described in Maintaining without updating translations.

5.8.5 Technical background

A number of Python scripts handle a part of the documentation translation process. All scripts used to maintain the translations are located in ‘scripts/auxiliar/’.

• ‘check_translation.py’ – show diff to update a translation,
• ‘texi-langutils.py’ – quickly and dirtily parse Texinfo files to make message catalogs and Texinfo skeleton files,
• ‘texi-skeleton-update.py’ – update Texinfo skeleton files,
• ‘update-snippets.py’ – synchronize ly snippets with those from English docs,
• ‘translations-status.py’ – update translations status pages and word counts in the file you are reading,
• ‘tely-gettext.py’ – gettext node names, section titles and references in the sources; WARNING only use this script once for each file, when support for "makeinfo –html" has been dropped.

Other scripts are used in the build process, in ‘scripts/build/’:

• ‘mass-link.py’ – link or symlink files between English documentation and documentation in other languages.

Python modules used by scripts in ‘scripts/auxiliar/’ or ‘scripts/build/’ (but not by installed Python scripts) are located in ‘python/auxiliar/’:

• ‘manuals_definitions.py’ – define manual names and name of cross-reference Texinfo macros,
• ‘buildlib.py’ – common functions (read piped output of a shell command, use Git),
• ‘postprocess_html.py’ (module imported by ‘www_post.py’) – add footer and tweak links in HTML pages.

And finally

• ‘python/langdefs.py’ – language definitions module

6. Website work

6.1 Introduction to website work

The website is not written directly in HTML; instead, the source is Texinfo, which is then generated into HTML, PDF, and Info formats. The sources are

Documentation/web.texi
Documentation/web/*.texi

Unless otherwise specified, follow the instructions and policies given in Documentation work. That chapter also contains a quick introduction to Texinfo; consulting an external Texinfo manual should be not necessary.

Exceptions to the documentation policies

• Sectioning: the website only uses chapters and sections; no subsections or subsubsections.
• @ref{}s to other manuals (@ruser, @rlearning, etc): you can’t link to any pieces of automatically generated documentation, like the IR or certain NR appendices.
• The bibliography in Community->Publications is generated automatically from ‘.bib’ files; formatting is done automatically by ‘texi-web.bst’.
• …
• For anything not listed here, just follow the same style as the existing website texinfo files.

6.2 Uploading and security

Overall idea

To reduce the CPU burden on the shared host (as well as some security concerns), we do not compile all of LilyPond. The website build process runs texi2html, but all media files (be they graphical lilypond output, photos of people, or pdfs) are copied from the $LILYPOND_WEB_MEDIA_GIT repository.

All scripts and makefiles used for the website build are run from a “trusted” copy. Any modification to those files in git needs a human to review the changes (after they have been made in git) before they are used on the server.

Building the website (quick local)

Initial setup: make sure that you have the environment variables $LILYPOND_GIT, $LILYPOND_BUILD_DIR and $LILYPOND_WEB_MEDIA_GIT set up correctly. For more information, see Environment variables.

Once that is done,

cd $LILYPOND_BUILD_DIR
make website

The website is in ‘out-website/website/index.html’.

Building the website (exactly as on the server)

Setting up (exactly as on the server)

Initial setup: you still need $LILYPOND_GIT and $LILYPOND_WEB_MEDIA_GIT.

Once that’s done, create:

mkdir -p $HOME/lilypond/
mkdir -p $HOME/lilypond/bin/
mkdir -p $HOME/lilypond/cron/
mkdir -p $HOME/lilypond/trusted-scripts/

The add these files to ‘$HOME/lilypond/bin/’:

Update git repositories:

### update-git.sh
#!/bin/sh
cd $LILYPOND_GIT
git fetch origin
git merge origin/master
cd $LILYPOND_WEB_MEDIA_GIT
git fetch origin
git merge origin/master

Check for any updates to trusted scripts / files:

### check-git.sh
#!/bin/sh
DEST=$HOME/lilypond/trusted-scripts
diff -u $DEST/website.make \
  $LILYPOND_GIT/make/website.make
diff -u $DEST/lilypond-texi2html.init \
  $LILYPOND_GIT/Documentation/lilypond-texi2html.init
diff -u $DEST/extract_texi_filenames.py \
  $LILYPOND_GIT/scripts/build/extract_texi_filenames.py
diff -u $DEST/create-version-itexi.py \
  $LILYPOND_GIT/scripts/build/create-version-itexi.py
diff -u $DEST/create-weblinks-itexi.py \
  $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py
diff -u $DEST/mass-link.py \
  $LILYPOND_GIT/scripts/build/mass-link.py
diff -u $DEST/website_post.py \
  $LILYPOND_GIT/scripts/build/website_post.py
diff -u $DEST/bib2texi.py \
  $LILYPOND_GIT/scripts/build/bib2texi.py
diff -u $DEST/langdefs.py \
  $LILYPOND_GIT/python/langdefs.py
diff -u $DEST/lilypond.org.htaccess \
  $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess
diff -u $DEST/website-dir.htaccess \
  $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess

If the changes look ok, make them trusted:

### copy-from-git.sh
#!/bin/sh
DEST=$HOME/lilypond/trusted-scripts
cp $LILYPOND_GIT/make/website.make \
  $DEST/website.make
cp $LILYPOND_GIT/Documentation/lilypond-texi2html.init \
  $DEST/lilypond-texi2html.init
cp $LILYPOND_GIT/scripts/build/extract_texi_filenames.py \
  $DEST/extract_texi_filenames.py
cp $LILYPOND_GIT/scripts/build/create-version-itexi.py \
  $DEST/create-version-itexi.py
cp $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py \
  $DEST/create-weblinks-itexi.py
cp $LILYPOND_GIT/scripts/build/mass-link.py \
  $DEST/mass-link.py
cp $LILYPOND_GIT/scripts/build/website_post.py \
  $DEST/website_post.py
cp $LILYPOND_GIT/scripts/build/bib2texi.py \
  $DEST/bib2texi.py
cp $LILYPOND_GIT/python/langdefs.py \
  $DEST/langdefs.py
cp $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess \
  $DEST/lilypond.org.htaccess
cp $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess \
  $DEST/website-dir.htaccess

Build the website:

### make-website.sh
#!/bin/sh
DEST=$HOME/web/
BUILD=$HOME/lilypond/build-website
mkdir -p $BUILD
cd $BUILD
cp $HOME/lilypond/trusted-scripts/website.make .

make -f website.make WEBSITE_ONLY_BUILD=1 website
rsync -raO $BUILD/out-website/website/ $DEST/website/
cp $BUILD/out-website/pictures $DEST
cp $BUILD/out-website/.htaccess $DEST

Then in the ‘cronjob/’ directory, put the cronjob to automate the trusted portions:

# website-rebuild.cron
LILYPOND_GIT=   ... fill this in
LILYPOND_WEB_MEDIA_GIT=   ... fill this in

11 * * * * $HOME/lilypond/trusted-scripts/update-git.sh >/dev/null 2>&1
22 * * * * $HOME/lilypond/trusted-scripts/make-website.sh >/dev/null 2>&1

As the final stage of the setup, run your copy-from-git.sh script, assuming that you trust the current state of scripts in lilypond git.

Normal maintenance

When there is a change to the build scripts and/or website makefile, log in to the server (or your own home machine if you’re testing this there), and do

update-git.sh
check-git.sh

After reviewing the changes carefully, you can update the trusted scripts with copy-from-git.sh.

Building the website (exactly as on the server)

Run make-website.sh; the final version ends up in ‘$HOME/web/’.

On the actual server, the website is generated hourly by user graham the host lilypond.org. You can set up the cronjob by doing:

crontab $HOME/lilypond/website-rebuild.cron

Initial setup for new users on actual serve

You should symlink your own ‘~/lilypond/’ to ‘~graham/lilypond/’

If this directory does not exist, make it. Git master should go in ‘~/lilypond/lilypond-git/’ but make sure you enable:

git config core.filemode false

If you have created any files in ‘~graham/lilypond/’ then please run:

chgrp lilypond ~graham/lilypond/ -R
chmod 775 ~graham/lilypond/ -R

Additional information

Some information about the website is stored in ‘~graham/lilypond/*.txt’; this information should not be shared with people without trusted access to the server.

6.3 Debugging website and docs locally

• Install apache2, or any other http server. These instructions assume that you also enable mod_userdir, and use $HOME/public_html as the location.
• Build the online docs and website:

  make WEB_TARGETS="offline online" doc
  make website
  

  This will make all the language variants of the website. To save a little time, just the English version can be made with the command make WEB_LANGS='' website or the English and (for example) the French with make WEB_LANGS='fr' website.

• Move the built stuff into those directories. It’s highly recommended to have your build dir and www dir on the same partition. (make $HOME/public_html/ a symlink if necessary)

  mv out-website/website/ $HOME/public_html
  mv $HOME/public_html/website/pictures $HOME/public_html/
  mkdir -p $HOME/public_html/doc/v2.13/
  mv out-www/online-root/* $HOME/public_html/doc/v2.13/
  

6.4 Translating the website

As it has much more audience, the website should be translated before the documentation; see Translating the documentation.

In addition to the normal documentation translation practices, there are a few additional things to note:

• Build the website with:

  make website
  

  however, please note that this command is not designed for being run multiple times. If you see unexpected output (mainly the page footers getting all messed up), then delete your ‘out-website’ directory and run make website again.

• Some of the translation infrastructure is defined in python files; you must look at the ### translation data sections in:

  scripts/build/create-weblinks-itexi.py
  scripts/build/website_post.py
  

• Translations are not included by default in make website. To test your translation, edit the WEB_LANGUAGES = line in ‘python/langdefs.py’. You will need to copy this updated script to $LILYPOND_BUILD_DIR/python/out.

  Do not submit a patch to add your language to this file unless make website completes with fewer than 5 warnings.

• Links to manuals are done with macros like @manualDevelLearningSplit. To get translated links, you must change that to @manualDevelLearningSplit-es (for es/Spanish translations, for example).

7. LSR work