Fork me on GitHub

shifter - blazingly fast yui builder

Jump to Table of Contents

welcome to shifter 0.4.2!

shifter

what is shifter?

The purpose of this project is to replace YUI's use of our old ant Builder.

We have out grown our old builder, so it was time to build a new one!

installation and usage

  1. download and install node.js
  2. run npm -g install shifter
  3. run shifter from inside the of a module inside the yui3 git repo or the yui3-gallery repo.

command line arguments

below is a simple list of commands that shifter supports.

$ shifter -h

blazingly fast builds with shifter@0.4.2

pass no arguments and shifter will build the module from the current directory

   -v/--version            show version
   -h/--help               show this stuff
   -m/--modules [module]   limit the modules to build (array: -m foo -m bar)
   --lint [preferred|defaults|strict] (preferred is the default) lint mode: https://github.com/yui/yui-lint
   --strict                add "use strict" to module wrapper
   -c/--config [file]      specify a config file name
   --ant                   parse the ant files and create a build.json but do not build
   --list                  List the builds and rollups from the build.json file
   --no-exec               Do not run pre/postbuild or pre/post execs
   --walk                  Walk the current directory and shift all builds. (cd yui3/src && shifter --walk)
                               -m/--modules also supported here for filtering
                               --no-progress to show the dots instead of the progress bar
   --watch                 Watch the current module and rebuild on file change (if meta file, a loader build will launch)
                               --quiet to mute stdout from sub build
                               all other build options accepted here: (--strict, --lint, etc)
   --jsstamp/--no-jsstamp  Should it stamp the JS with the YUI.add wrapper, defaults to --stamp
   --istanbul              Use Istanbul code coverage instead of YUITest for coverage build
Experimental Options:
   --semi                  Toggle on the strict semicolon checking in Uglify
   --cache/--no-cache      Cache the results of the build and bail if building for no reason, defaults to --no-cache
   --cache-file <path>     File to store build cache, defaults to $CWD/.shifter_meta
   --fail                  Fail the build if lint fails
   --compressor            Use YUI Compressor instead of uglify
   --no-lint               Skip JSlint, you better know what you are doing!

More information about experimental options.

what does it do?

shifter will parse your current *.properties files and convert them into a build.json file that it can process. It only imports the relevant settings required to build the module.

It does not import module meta-data (see the meta-data for more information

Instead, shifter parses the meta-data from the modules meta/*.json files and uses that instead. So you don't have to declare your meta-data in more than one place now.

migrating to shifter

shifter is designed to work side by side with our current builder (for now) so you don't have to switch over to using it fully if it doesn't work properly for you. Just don't delete your *.properties files until you are sure that Shifter builds your module properly. If it doesn't, file a ticket and we'll get it fixed up ASAP.

shifter will read a build.json file if it exists, if one doesn't and it finds a *.properties file it will generate the build.json from them. So if you have issues with the build, just delete the build.json file and have Shifter regenerate it after your issue is fixed.

watching and building

shifter can watch your module for changes and build for you. It will only watch files in the ./js, ./css, ./assets and ./meta directories. If a file is changed, it will rebuild the current module. If a meta file is changes, Loader will also be built (requires latest YUI source code).

skin handling

shifter will attempt to process skins the way the old builder used to do them, but with the same side-effect/bug of creating skins for all submodules even if they don't have a skin.

shifter now supports a valid way to handle this, if you "namespace" your assets under a "module" directory below assets, shifter will do the right thing.

Here is an example directory structure, it's easier to read than to explain.

Code Coverage

When shifter builds a module, it auto generates a -coverage file generated with YUITest Code Coverage

If you pass the --istanbul option, shifter will use the new Istanbul Code Coverage tool.

meta-data

One of the goals of shifter was to remove the need for duplicating Loader meta-data. This includes:

component.requires=classnamemanager, pjax-base
component.use=foo,bar,baz
component.skinnable=true

These properties were also being redefined in the modules meta/<module>.json file to be included when Loader was built.

When shifter parses the build.json file, it will attempt to gather Loader meta-data from the corresponding meta/*.json files and munge them together into the data it needs to properly build the module.

You can add your own to the build file if you do not have a meta file to parse: 

{
    "name": "foobar",
    "builds": {
        "module": {
            "jsfiles": [
                "./js/foo.js"
            ],
            "config": {
                "use": [
                    "yui-base",
                    "get",
                    "features",
                    "intl-base",
                    "yui-log",
                    "yui-later",
                    "loader-base",
                    "loader-rollup",
                    "loader-yui3"
                ]
            }
        }
    }
}

Note: If you provide a config object and you have Loader meta-data, shifter will attempt to merge them together, so your results may vary on this.

build.json reference

For an object reference jump to the table below

Simple build.json example

{
    "name": "yql",
    "builds": {
        "yql": {
            "jsfiles": [
                "yql.js"
            ]
        }
    }
}

A more complex build.json file (a truncated version of the yui/build.json file)

{
    "name": "yui",
    "prebuilds": [
        "get",
        "loader"
    ],
    "postbuilds": [
        "simpleyui"
    ],
    "exec": [
        "./scripts/build.js"
    ],
    "builds": {
        "yui-log": {
            "jsfiles": [
                "yui-log.js"
            ]
        },
        "yui-core": {
            "name": "yui-base",
            "replace": {
                "@YUI_CORE@": "['intl-base']"
            },
            "prependfiles": [
                "js/yui.js"
            ],
            "jsfiles": [
                "yui-base.js",
                "yui-object.js",
                "yui-ua.js",
                "alias.js"
            ]
        }
    },
    "rollups": {
        "yui-base": {
            "name": "yui",
            "replace": {
                "@YUI_CORE@": "['get', 'features', 'intl-base', 'yui-log', 'yui-later']"
            },
            "config": {
                "use": [
                    "yui-log",
                    "yui-later"
                ]
            },
            "files": [
                "yui-base",
                "yui-log",
                "yui-later"
            ],
            "build": {
                "prependfiles": [
                    "js/yui.js"
                ],
                "jsfiles": [
                    "yui-base.js",
                    "yui-lang.js",
                    "alias.js"
                ]
            }
        }
    }
}

build.json object reference

root properties

key value
name String the name of this module
shifter Object containing default shifter options: CLI options will override these. 
"shifter": {
    "jsstamp": false,
    "coverage": false,
    "lint": "defaults"
}
prebuilds* Array of modules that you want to build before this build (CWD ../ module) "prebuilds" [ "foo", "bar" ]`
postbuilds* Array same as prebuilds only executes after the build is complete.
exec* Array of scripts to execute before the build
postexec* Array of scripts to execute after the build
builds Object describes the module builds to perform
rollups Object describes the module rollups to perform after the build

* denotes that these properties can also be added to an individual build as well as globally.

global config file

shifter will walk up the working directory and search for the first .shifter.json file that it finds. It will then apply any configuration settings found in that config to the current shifter config. (described above). Below is an example .shifter.json: 

{
    "replace-yuiglobalvar": "FOO",
    "replace-yuivar": "F",
    "replace-version": "1.2.3.4",
    "lint": false,
    "coverage": false,
    "regex": "^.*?(?:logger|Y.log).*?(?:;|\\).*;|(?:\\r?\\n.*?)*?\\).*;).*;?.*?\\r?\\n|^.*?(?:\/\\*@DBG\\*\/).*\\r?\\n"
}

You can force shifter to ignore this file with the --no-global-config option to the cli. CLI options will override the options in the config

builds properties

Properties available on the builds property.

key value
jsfiles Array of files under the js directory to concat (in this order)
cssfiles Array of files under the css directory to concat (in this order)
skinnable Boolean should a skin be built
regex String A regex to apply to the build to remove Y.log statements, the default is:
/^.*?(?:logger|Y.log).*?(?:;|\).*;|(?:\r?\n.*?)*?\).*;).*;?.*?\r?\n/mg
This string overrides the one defined in .shifter.json
An empty string here disable this behavior for that specific build.
prependfiles Array list of files to include before the concatted jsfiles|cssfiles
appendfiles Array same as prependfiles only after the concatted files
replace Object search the build for Object.key and replace with Object.value
config Object see meta-data
copy Array of Array's containig from and to:
"copy": [
    [ "./foo", "./bar" ],
    [ "./foobar", "./barfoo" ]
]

rollups properties

Properties available on the rollups property.

Note: The logic behind rollups changed in shifter, in the previous builder a rollup performed a full build on the files before rolling them up. This wasted valuable time and needed to be changed.

With shifter a default rollup is nothing more than stitching a few pre-built modules together before stamping them with a module stamp.

key value
replace Same as on builds
config Same as on builds
files Array of modules to rollup: "files": [ "foo", "bar" ] will look for build/foo/foo-debug.js and build/bar/bar-debug.js and roll them up
build Object perform this build first, then proceed with the rollup (same as a property under builds)

experimental options

From time to time I will add experimental options to shifter to allow for developers to test new features prior to turning them on.

build file caching

You can turn on build time caching with the --cache config option. This will write a cache file out with the MD5's of the build. If the MD5 matches one in the meta file, the build will abort and skip the remaining tasks (compressor, coverage, etc) to speed up development.

You can also use --cache-file [path] to specify a cache file instead of using a per module one. Defaults to $CWD/.shifter_meta.

failing build from lint issues

Pass --fail to fail the build if any lint errors occur.

strict semicolon checking in Uglify

The default is to not require strict semi colons with UglifyJS, for example:

var foo = {};

foo.bar = function () {
}

That last } should be };, in strict mode Uglify will throw without that semi colon.

skipping jslint

Some developers do not agree with JSLint, so this option is there for them to use whatever lint program they choose to use. As long as they use one!

using YUI Compressor

As of version 0.1.0 Shifter defaults to using UglifyJS as it's default compression utility. If there is an issue with compressing your module with Uglify, you can revert to using YUI Compressor with the --compressor flag.

Forcing lint to stderr

By default, lint warnings/errors are printed to stdout. But in the case that you want to trap that in logs or in a CI system, I've added a --lint-stderr config option which forces all lint output to stderr.

CLI Replacers

You can pass --replace-??=?? and shifter will attempt to replace these strings during the build.

You MUST use the = to tell nopt that you want to assign the value to the dynamic option.

Examples:

--replace-version=1.2.3 will replace @VERSION@ with 1.2.3
--replace-foo=bar will replace @FOO@ with bar
--replace-baz=bog will replace @BAZ@ with bog

Recursive Building

Adding --recursive to the --walk command will tell shifter to walk the directories recursively looking for build.json files.

Build Status