diff --git a/README.md b/README.md index c34e0c3e..1a0aed9e 100644 --- a/README.md +++ b/README.md @@ -1,19 +1,48 @@ -# DMS crowdin Maven Plugin +# Crowdin Maven Plugin -This plugin allows Maven projects to be translated using crowdin. It is based on [glandais' crowdin-maven plugin](https://github.com/glandais/crowdin-maven), but has been adapted for use with the [Digital Media Server project](https://github.com/DigitalMediaServer/DigitalMediaServer). +This Maven plugin synchronizes translation files between the local project and Crowdin using the Crowdin API. It was originally based on [glandais' crowdin-maven plugin](https://github.com/glandais/crowdin-maven), but has since been completely rewritten. The Maven project must be in a Git repository since Git branches are translated to Crowdin branches. -## Configuration +## Table of Contents +- [1. Configuration](#1-configuration) + - [1.1 Credentials](#11-credentials) + - [1.1.1 Example credentials configuration](#111-example-credentials-configuration) + - [1.1.2 Settings template](#112-settings-template) + - [1.2 Project configuration](#12-project-configuration) + - [1.2.1 Skeleton project configuration](#121-skeleton-project-configuration) + - [1.2.2 Parameter description](#122-parameter-description) + - [1.2.2.1 `statusFile` parameter description](#1221-statusfile-parameter-description) + - [1.2.2.2 `translationFileSet` parameter description](#1222-translationfileset-parameter-description) + - [1.2.2.3 `conversion` parameter description](#1223-conversion-parameter-description) + - [1.2.2.4 `escapeQuotes` options](#1224-escapequotes-options) + - [1.2.2.5 `updateOption` options](#1225-updateoption-options) + - [1.2.2.6 Crowdin file types](#1226-crowdin-file-types) + - [1.2.2.7 Crowdin placeholders](#1227-crowdin-placeholders) + - [1.2.2.8 Additional `targetFileName` placeholders](#1228-additional-targetfilename-placeholders) + - [1.2.3 Example project configuration](#123-example-project-configuration) +- [2. Using the plugin](#2-using-the-plugin) + - [2.1 Pushing strings for translation to Crowdin](#21-pushing-strings-for-translation-to-crowdin) + - [2.2 Getting translations from Crowdin](#22-getting-translations-from-crowdin) + - [2.3 Cleaning the intermediate folder](#23-cleaning-the-intermediate-folder) -To access crowdin this plugin needs a project identifier and an API key from crowdin. This is achieved by specifying a ```server``` in Maven configuration. The API key needs to be kept private, so the best place to put this is in the Maven user settings. This can be achieved by adding a server to your ```~/.m2/settings.xml``` like shown in the example below. If you don't have a ```~/.m2/settings.xml``` file, a template is provided later in this README which can be copy and pasted into an empty xml file. +## 1. Configuration +This plugin needs configuration to know what files to handle and how. It is meant to be executed manually using Maven, but it would require too many command-line arguments to be practical if the configuration wasn't stored somewhere. The project's `pom.xml` is therefore used to store most of the configuration, while the Crowdin credentials is stored in Maven's settings, preferably in `~/.m2/settings.xml`. This way the configuration is checked in to the Git repository while the credentials are kept in the local file system for those that are authorized to synchronize translations with Crowdin. + +The `pom.xml` configuration for this plugin doesn't affect the Maven build process by default, as the only `phase` it binds to is the `clean` phase. It is possible to bind one or more `goal` from this plugin to specific build phases, but it's hard to see how that could be useful. Running the plugin without having configured the `pom.xml` and the credentials will result in an error. + +### 1.1 Credentials + +To access Crowdin the project identifier and API key are needed. This is configured by specifying a ```server``` in Maven's configuration. The API key needs to be kept private, so the best place to put it is usually in Maven's user settings. This can be achieved by adding a server to your ```~/.m2/settings.xml``` as shown in the example below. If you don't have a ```~/.m2/settings.xml``` file, a template that can be pasted into an empty file [is provided](#112-settings-template). + +#### 1.1.1 Example credentials configuration ```xml - crowdin-dms - DigitalMediaServer + crowdin + DigitalMediaServer API key @@ -22,7 +51,40 @@ To access crowdin this plugin needs a project identifier and an API key from cro ``` -Further parameters need to be configured in your project's ```pom.xml``` so that they are available when requested by the plugin. This plugin doesn't bind to any ```phases``` and must be executed manually, so the changes to the ```pom.xml``` don't affect the build process itself. Below is a typical configuration. +#### 1.1.2 Settings template + +If you're missing the file ```~/.m2/settings.xml```, you can copy and paste the template below: + +```xml + + + + + + + + + crowdin + Your project identifier + Your API key + + + + + + + +``` + +### 1.2 Project configuration + +The project configuration is stored in the `pom.xml` of the project. A detailed description is given below. + +#### 1.2.1 Skeleton project configuration + +Here is a skeleton project configuration showing the location of all configuration options: ```xml @@ -36,16 +98,73 @@ Further parameters need to be configured in your project's ```pom.xml``` so that org.digitalmediaserver crowdin-maven-plugin - LATEST + ... - ${project} - ${project.basedir}/src/main/resources/i18n - ${project.basedir}/extras/crowdin - ${project.basedir}/src/main/resources/languages.properties - crowdin-dms - messages.properties - Digital Media Server - Digital Media Server + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -58,65 +177,248 @@ Further parameters need to be configured in your project's ```pom.xml``` so that ``` -## Parameter description +#### 1.2.2 Parameter description -* ```languageFilesFolder``` - The folder where the translation ```.properties``` files are located. -* ```downloadFolder``` - A temporary folder used to store the files downloaded from crowdin. -* ```statusFile``` - The full path to the ```.properties``` file where the translation status should be written. -* ```crowdinServerId``` - The ```id``` of the Maven configured ```server``` to be used by the plugin for crowdin authentication. -* ```pushFileName``` - The name of the file located in ```languageFilesFolder``` that is the _base language_ file that should be uploaded to crowdin during ```push```. -* ```pushFileTitle``` - The title to be associated with ```pushFileName``` on crowdin. -* ```projectName``` - The crowdin project name. To avoid pushing the wrong file to crowdin, this must match both the ```pom.xml``` project name and the crowdin project name for the given crowdin project identifier for ```push``` to be executed. -* ```rootBranch``` - The git branch that should be considered root on crowdin (that is, not exist in a branch folder). The default value is ```master```. This parameter can be specified in ```pom.xml```, from the command line with ```-DrootBranch=``` or left to it's default. Any git branch not matching this parameter will push to and fetch from a branch folder at crowdin. +|Name|Type|Req.|Default|Description| +|--|:--:|:--:|:--:|--| +|`comment`|String|No| * |The comment to add to the top of downloaded translation files. If defined, this parameter acts as a default for all `translationFileSets` and `statusFiles`.| +|`confirm`|String|`push`| |This is required to be `true` to use the `push` goal. This parameter can be overridden on the command line with `-Dconfirm`. Any strings that exist on Crowdin but don't exist in the uploaded files will have all their translations deleted on Crowdin when pushed. As such, it's important to make sure that a push is intended. Although this parameter can be set to `true` in `pom.xml`, it is recommended not to. That way, adding `-Dconfirm` to the command line is required to be able to push.| +|`crowdinServerId`|String|Yes| |The `id` of the Maven configured `server` to be used for Crowdin authentication.| +|`downloadFolder`|String|Yes| |The intermediate folder used to store the downloaded files.| +|`escapeQuotes`|Integer|No|`0`|The `escape_quotes` [Crowdin API parameter](https://support.crowdin.com/api/add-file/). See [separate definition](#1224-escapequotes-options). This is not used by this plugin, and is merely passed on to Crowdin. If defined, this parameter acts as a default for all `translationFileSets` and `statusFiles`.| +|`lineSeparator`|String|No| |An alternative line separator to apply to the downloaded files, for example `\n` or `\r\n`. If defined, this parameter acts as a default for all `translationFileSets` and `statusFiles`.| +|`projectName`|String|`push`| |This is required to use the `push` goal. The value must match the project name defined in `pom.xml`. It is a safety check to make sure you don't push to the wrong project if the configuration has been copied from another project.| +|`rootBranch`|String|No|`master`|The Git branch that should be considered the root on Crowdin (that is; not exist in a branch folder). This parameter can be overridden on the command line with `-DrootBranch=`. Any local Git branch not matching this parameter will push to and fetch from a branch folder at Crowdin.| +|`statusFiles`|List|No| |A list of one or more `statusFile` elements. A `statusFile` element represents a local status file. This is a file a file in either `properties` or `xml` format, whose content is the output of the `status` [Crowdin API method](https://support.crowdin.com/api/status/). The file contains basic information about the state of the translations per language for all files in total. Crowdin doesn't allow getting the status per file, so having more than one status file for a project would serve little purpose. See [separate definition](#1221-statusfile-parameter-description).| +|`translation` `FileSets`|List|Yes| |A list of one or more `translationFileSet` elements. A `translationsFileSet` element represents a local *base language file* and its set of corresponding translations in other languages. It also represents a single file on Crowdin. Only the *base language file* will be uploaded to Crowdin, and only the corresponding translated language files will be downloaded. See [separate definition](#1222-translationfileset-parameter-description).| +|`updateOption`|Enum|No|Delete|The `update_option` [Crowdin API parameter](https://support.crowdin.com/api/update-file/). See [separate definition](#1225-updateoption-options). This is not used by this plugin, and is merely passed on to Crowdin. If defined, this parameters acts as a default for all `translationFileSets` and `statusFiles`.| -## Using the plugin +`*` The default comment is `This file has been generated automatically, modifications will be overwritten. If you'd like to change the content, please do so at Crowdin.` -Given that the parameters are configured correctly, you can execute a goal with: +##### 1.2.2.1 `statusFile` parameter description -```mvn dms-crowdin: [-D=]``` +|Name|Type|Req.|Default|Description| +|--|:--:|:--:|:--:|--| +|`addComment`|Boolean|No|`true`|Sets whether or not a comment should be added at the top of the status file when deploying.| +|`comment`|String|No| |The comment to add at the top of the status file. If defined, this parameter overrides the corresponding parameter from the "root" plugin configuration.| +|`encoding`|String|No|UTF-8|The encoding to use for the status file. The default is `ISO 8859-1` if the file type is `properties`.| +|`escapeUnicode`|Boolean|No|`true`|This only applies if the type is `properties`. If `true`, Unicode characters will be encoded in the form `\u` where `` is the hexadecimal Unicode code point.| +|`lineSeparator`|String|No| |An alternative line separator to apply to the status file, for example `\n` or `\r\n`. If defined, this parameter overrides the corresponding parameter from the "root" plugin configuration.| +|`sortLines`|Boolean|No|`true`|This only applies if the type is `properties`. If `true`, this will sort the lines of the status file by key when deploying. The keys are "grouped" by `.`. Integer groups are sorted numerically, all other groups are sorted lexicographically. When comparing integer groups with non-integer groups, integer groups are sorted last.| +|`targetFile`|String|Yes| |The full path to this status file.| +|`type`|Enum|No|`properties`|The file type to use when deploying the status file. Valid values are `properties` and `xml`.| +|`conversions`|List|No| |A list of one or more `conversion` elements. A `conversion` element represents a "find and replace operation" that will be executed during deployment. It applies to the language codes only. See [separate definition](#1223-conversion-parameter-description).| -This plugin requires git to be installed to work, and will automatically look up the current git branch and use that as the crowdin branch unless the current git branch matches to ```rootBranch``` parameter. If no git branch is found (e.g. checked out to a tag) all goals but ```deploy``` will fail. For further information about versions management and branches on crowdin, see the [crowdin documentation] (https://support.crowdin.com/articles/versions-management/). +##### 1.2.2.2 `translationFileSet` parameter description -The goals are explained below: +|Name|Type|Req.|Default|Description| +|--|:--:|:--:|:--:|--| +|`addComment`|Boolean|No|`true`|Sets whether or not a comment should be added at the top of translation files when deploying.| +|`baseFileName`|String|Yes| |The name or path to the "base language file" that should be uploaded to Crowdin for translation, relative to `languageFilesFolder`.| +|`comment`|String|No| |The comment to add at the top of translation files. If defined, this parameter overrides the corresponding parameter from the "root" plugin configuration.| +|`commentTag`|String|No|`#`|The character (sequence) to use if a comment is added during the `deploy` goal. This is not used for `Properties`, `HTML` or `XML` files.| +|`crowdinPath`|String|No| |The path from the root or branch root folder to the location of this set of files on Crowdin.| +|`encoding`|String|No|`UTF-8`|The encoding to use for the deployed translation files. The default is `ISO 8859-1` if the file type is `properties`.| +|`escapeQuotes`|Integer|No|`0`|The `escape_quotes` [Crowdin API parameter](https://support.crowdin.com/api/add-file/). See [separate definition](#1224-escapequotes-options). This is not used by this plugin, and is merely passed on to Crowdin. If defined, this parameter overrides the corresponding parameter from the "root" plugin configuration.| +|`escapeUnicode`|Boolean|No|`true`|This only applies if the type is `properties`. If `true`, Unicode characters will be encoded in the form `\u` where `` is the hexadecimal Unicode code point.| +|`fileName` `WhenExported`|String|Yes| |This is used by Crowdin to generate filenames during export, and is the string specified under "Resulting file name when exported" in the file settings at Crowdin. It is also used by this plugin to parse the filenames exported from Crowdin to recognize placeholders. The placeholder codes are defined by Crowdin and are also listed in [this table](#1227-crowdin-placeholders).| +|`language` `FilesFolder`|String|Yes| |The folder where this set of files is located.| +|`lineSeparator`|String|No| |An alternative line separator to apply to the translation files, for example `\n` or `\r\n`. If defined, this parameter overrides the corresponding parameter from the "root" plugin configuration.| +|`sortLines`|Boolean|No|`true`|This only applies if the type is `properties`. If `true`, this will sort the lines of the translation files by key when deploying. The keys are "grouped" by `.`. Integer groups are sorted numerically, all other groups are sorted lexicographically. When comparing integer groups with non-integer groups, integer groups are sorted last.| +|`targetFileName`|String|No| |The file path relative to `languageFilesFolder` to use when deploying the translation files. If left blank, the path exported by Crowdin (`fileNameWhenExported`) is used. Any placeholders used in `fileNameWhenExported` can be used. In addition, [these placeholders](#1228-additional-targetfilename-placeholders) can be used independently of what is used in `fileNameWhenExported`.| +|`title`|String|No| |The title of this file as it should appear to translators at Crowdin.| +|`type`|Enum|No|Auto|The file type to use both when uploading to Crowdin and when processing files during the `deploy` goal. See [separate definition](#1226-crowdin-file-types). If not specified, auto-detection will be attempted based on the file extension.| +|`updateOption`|Enum|No|Delete|The `update_option` [Crowdin API parameter](https://support.crowdin.com/api/update-file/). See [separate definition](#1225-updateoption-options). This is not used by this plugin, and is merely passed on to Crowdin. If defined, this parameter overrides the corresponding parameter from the "root" plugin configuration.| +|`conversions`|List|No| |A list of one or more `conversion` elements. A `conversion` element represents a "find and replace operation" that will be executed during deployment. It applies to placeholders only and must match the complete placeholder. The content of any placeholders that match will be replaced. See [separate definition](#1223-conversion-parameter-description).| +|`excludes`|List|No| |A list of one or more translation files to exclude. This is a basic filter that works on the file names exported from Crowdin, before any conversions are performed. It works like most file system searches, where the only wildcard characters are `*` and `?`.| +|`includes`|List|No| |A list of one or more translation files to include. This is a basic filter that works on the file names exported from Crowdin, before any conversions are performed. It works like most file system searches, where the only wildcard characters are `*` and `?`. If one or more `include` elements are defined, any paths that aren't included are excluded (it becomes a white-list).| -### Pushing strings for translation to crowdin +##### 1.2.2.3 `conversion` parameter description -*Goal* | *Command* | *Description* ----- | ------- | ----------- -**push** | ```mvn dms-crowdin:push -Dconfirm=true``` | Upload the _base language_ file to on crowdin. Any strings already on crowdin that's missing in the uploaded file is deleted from crowdin with all corresponding translations. Because of this, an extra argument ```confirm=true``` is required for ```push```. +|Name|Type|Req.|Description| +|--|:--:|:--:|--| +|`from`|String|Yes|The value to match against any placeholder| +|`to`|String|Yes|The value to replace the placeholder content with| -### Getting translations from crowdin +##### 1.2.2.4 `escapeQuotes` options -*Goal* | *Command* | *Description* ----- | ------- | ----------- -**build** | ```mvn dms-crowdin:build``` | Ask crowdin to build a downloadable zip file containing all the latest translations. Unpaid projects can only build once every 30 minutes via the API, but it's possible to build from the crowdin web interface at any time. The API replies with status ```skipped``` both if there are no changes since the last build and if the previous build was less than 30 minutes ago, so there's no way to tell the two apart. -**fetch** | `mvn dms-crowdin:fetch` | Download and extract the last built zip file from crowdin to ```downloadFolder```. -**deploy** | `mvn dms-crowdin:deploy` | Copy the downloaded files from ```downloadFolder``` and into their intended locations in accordance with ```languageFilesFolder``` and ```statusFile```. -**pull** | ```mvn dms-crowdin:pull``` | Perform ```build```, ```fetch``` and ```deploy``` in sequence. This is a convenience goal combining the individual steps to get the latest translations from crowdin copied into your local project. +| Code | Description | +|--|--| +|0|Do not escape single quote| +|1|Escape single quote by another single quote| +|2|Escape single quote by backslash| +|3|Escape single quote by another single quote only in strings containing variables (`{0}`)| -## ```settings.xml``` template +##### 1.2.2.5 `updateOption` options -If you're missing the file ```~/.m2/settings.xml```, you can copy and paste the template below: +| Code | Description | +|--|--| +|`delete_translations`|Delete translations of changed strings| +|`update_as_unapproved`|Preserve translations of changed strings but remove validations of those translations if they exist| +|`update_without_changes`|Preserve translations and validations of changed strings| + +##### 1.2.2.6 Crowdin file types + +| Code | Description | +|--|--| +|`auto`|Try to detect file type by extension or MIME type| +|`android`|Android| +|`macosx`|Mac OS X / iOS| +|`resx`|.NET, Windows Phone| +|`properties`|Java| +|`gettext`|GNU GetText| +|`yaml`|Ruby On Rails| +|`php`|Hypertext Preprocessor| +|`json`|Generic JSON| +|`xml`|Generic XML| +|`ini`|Generic INI| +|`rc`|Windows Resources| +|`resw`|Windows 8 Metro| +|`resjson`|Windows 8 Metro| +|`qtts`|Nokia Qt| +|`joomla`|Joomla localizable resources| +|`chrome`|Google Chrome Extension| +|`dtd`|Mozilla DTD| +|`dklang`|Delphi DKLang| +|`flex`|Flex| +|`nsh`|NSIS Installer Resources| +|`wxl`|WiX Installer| +|`xliff`|XLIFF| +|`html`|HTML| +|`haml`|Haml| +|`txt`|Plain Text| +|`csv`|Comma Separated Values| +|`md`|Markdown| +|`flsnp`|MadCap Flare| +|`fm_html`|Jekyll HTML| +|`fm_md`|Jekyll Markdown| +|`mediawiki`|MediaWiki| +|`docx`|Microsoft Office, OpenOffice.org Documents, Adobe InDesign Adobe FrameMaker| +|`sbv`|Youtube `.sbv`| +|`vtt`|Video Subtitling and WebVTT| +|`srt`|SubRip `.srt`| + +##### 1.2.2.7 Crowdin placeholders +| Code | Description | +|--|--| +|`%language%`|Language name (e.g. Ukrainian).| +|`%two_letters_code%`|Language code `ISO 639-1` (i.e. `uk`).| +|`%three_letters_code%`|Language code `ISO 639-2/T` (i.e. `ukr`).| +|`%locale%`|Locale (i.e. `uk-UA`).| +|`%locale_with_underscore%`|Locale (i.e. `uk_UA`).| +|`%android_code%`|Android Locale identifier used to name `values-` folders.| +|`%osx_code%`|macOS Locale identifier used to name `.lproj` folders.| +|`%osx_locale%`|macOS Locale used to name translated resources (i.e. `uk`, `zh_Hans`).| +|`%original_file_name%`|Original file name.| +|`%file_name%`|File name without extension.| +|`%file_extension%`|Original file extension.| +|`%original_path%`|Use parent folders' names in your project to build the file path in the resulting archive.| +##### 1.2.2.8 Additional `targetFileName` placeholders +| Code | Description | +|--|--| +|`%crowdin_code%`|Crowdin language code (i.e `en-GB` or `da`).| +|`%crowdin_code_with_underscore%`|Crowdin language code with underscore (i.e `en_GB` or `da`).| +|`%shortest_iso639_code%`|The shortest `ISO 639` language code (i.e `en` or `ceb`).| +|`%language%`|Language name (e.g. Ukrainian).| +|`%two_letters_code%`|Language code `ISO 639-1` (i.e. `uk`).| +|`%three_letters_code%`|Language code `ISO 639-2/T` (i.e. `ukr`).| + +#### 1.2.3 Example project configuration ```xml - - - - - - - - - crowdin-dms - DigitalMediaServer - API key - - - - - - - + + + + + + + + + + org.digitalmediaserver + crowdin-maven-plugin + ... + + Digital Media Server + crowdin + ${project.basedir}/extras/crowdin + + + Digital Media Server + ${project.basedir}/src/main/resources/i18n + messages.properties + messages_%locale_with_underscore%.properties + messages_%crowdin_code_with_underscore%.properties + update_as_unapproved + + + es-ESes + + + + + + + + + + + ${project.basedir}/src/main/resources/languages.properties + + + es-ESes + + + + + + + + + + + + + + + ``` + +## 2. Using the plugin + +This plugin requires Git, and will automatically look up the current Git branch. Unless the current Git branch matches the `rootBranch` parameter, all operation will be performed against a branch with the same name on Crowdin. If no Git branch is found (e.g. checked out to a tag or detached) all goals except `deploy` will fail. This allows translation of a branch while it's being worked on and merging of the translations at Crowdin when the branch is merged. For further information about versions management and branches on Crowdin, see the [Crowdin documentation](https://support.crowdin.com/articles/versions-management/). + +If the configuration is correct, you can execute a goal with: + +`mvn crowdin-maven-plugin: [-D=]` + +The prefix or alias `crowdin` will be resolved to this plugin, unless another plugin configured in the same `pom.xml` use the same alias and registers it first. That means that it is very likely that you can instead execute a goal with: + +`mvn crowdin: [-D=]` + +This short form will be used in the descriptions below. + + + +### 2.1 Pushing strings for translation to Crowdin + +|*Goal*|*Command*|*Description*| +|--|--|--| +|**push** | `mvn crowdin:push -Dconfirm=true` | Uploads the _base language_ file to Crowdin. Any strings already present on Crowdin that's missing in the uploaded file will be deleted from Crowdin together with all corresponding translations. To avoid accidental pushes, an extra argument `confirm` is required for `push`.| + +### 2.2 Getting translations from Crowdin + +|*Goal* | *Command* | *Description*| +|--|--|--| +|**build** | `mvn crowdin:build` | Asks Crowdin to build a downloadable zip file containing all the latest translations. This file is used by the `fetch` goal, so it the zip file isn't up to date, `fetch` will download stale translations.

Unpaid projects can only build once every 30 minutes via the API, but it's possible to build from the Crowdin web interface at any time. Unfortunately, it's not possible to build branches from the Crowdin web interface. The API replies with status `skipped` both if there are no changes in the translations since the last build and if the previous build was less than 30 minutes ago, so there is no way to tell the two apart.| +|**fetch** | `mvn crowdin:fetch` | Downloads and extracts the last built zip file from Crowdin to `downloadFolder`.| +|**deploy** | `mvn crowdin:deploy` | Applies any transformations and deploys the files from `downloadFolder` into their intended locations as defined by the [translationsFileSets](#1222-translationfileset-parameter-description) and the [statusFiles](#1221-statusfile-parameter-description).| +|**pull** | `mvn crowdin:pull` | Executes `build`, `fetch` and `deploy` in sequence. This is a convenience goal combining the individual steps needed to get the latest translations from Crowdin built and deployed into your local project.| + +### 2.3 Cleaning the intermediate folder + +|*Goal* | *Command* | *Description*| +|--|--|--| +|**clean** | `mvn crowdin:clean` | Deletes all content in `downloadFolder`.| + +The `clean` goal also binds to the `clean` phase, which means that it's automatically executed when `mvn clean` is run.