files: list all required files for build tools

[desandro]

Since resolving main in #43, we've thrown a wrench in the gears for build tools, most prevelantly Wiredep. Wiredep needs a list of all the files to be included when you install the package. Previously, main was being widely used for this purpose. Now main should no longer lists all the files (The entry-point files necessary to use your package. One per filetype). We need another field.

These two proposals are my first shot at it. Your thoughts appreciated 🍉

Proposal 1: files flat list

files is an array of all the files required to use the package. Filepaths may include globs.

files differs from main in that main only lists single files per filetype, whereas files should include every file.

We recommend listing source files in files, rather than compiled distribution files.

"files": [
  "js/motion.js",
  "js/run.js",
  "js/walk.js",
  "sass/motion.scss",
  "sass/run.scss",
  "sass/walk.scss",
  "img/motion.png",
  "img/walk.png",
  "img/run.png",
  "fonts/icons.woff2",
  "fonts/icons.woff"
]

Most likely you would list globs for multiple files of the same type.

"files": [
  "js/*.js",
  "sass/*.scss",
  "img/*.*",
  "fonts/*.*"
]

Pros: Simple specification, easier for developers to adopt. Don't have to worry on Bower spec side about asset-type specification. Cons: Aside from file extension, it's not clear what files do what. For example, a package make require icon.svg — is this file a font or should be it used as an image?

We leave it up to build tools on how they make use of the array. Some might only be concerned about .js and .css files. For globs, they'd have to expand the glob and do the logic on their side.

Proposal 2: asset-type object

Taking from #21

files is an object listing all the files required to use the package. files are listed as arrays for each type of asset.

• scripts: Script files like JavaScript .js or CoffeeScript .coffee
• styles: Style files like CSS .css, Sass .scss, or LESS .less
• images: ...
• fonts: ...
• data
• templates
• (Others — we'd have to determine this spec)

Filepaths may include globs. yada yada yada ....

"files": {
  "scripts": [
    "js/motion.js",
    "js/run.js",
    "js/walk.js",
  ],
  "styles": [
    "sass/motion.scss",
    "sass/run.scss",
    "sass/walk.scss",
  ],
  "images": [
    "img/motion.png",
    "img/walk.png",
    "img/run.png",
  ],
  "fonts": [
    "fonts/icons.woff2",
    "fonts/icons.woff"
  ]
}

Pro: Clear about what files do what. Cons: More complex. We'd have to figure out the list of types. Might be confusing files that do not have an explicit type, like if there's a video file and we don't have a videos field.

Fun fact: recommended way back in 2013 bower/bower#935 (comment)

---

files would live along side main, not overwrite it. I'm not sure if it should be required or part of the bower init flow at this point.

Currently, I prefer Proposal 1. Easier for authors and for Bower spec.

Related issues

• Changing bower.json main as entry-point files, not distribution files taptapship/wiredep#180
• bower.json not including css file in main twbs/bootstrap#16663
• Wiredep bug - item.js missing in bower.json main metafizzy/outlayer#33

[HeroBart]

So the new "files" will hold all source files, and "main" will hold compiled versions again?
If so, I'd vote for proposal 1, it's simple enough.

[kkirsche]

With proposal 2 I think you have the potential to accept valid MIME types as the category list. By doing this, it's not up to you to figure out what kinds of files people need and you offload that to a different organization where it is getting close monitoring. Personally, I prefer proposal 2 as I feel there is more clarity which would be useful to users who are new or inexperienced with a project.

[alber70g]

Does this mean that the dist/movement.css used in the example of the current spec can be in main? Because if that's the case, nothing really changes. Except the hard-exclude of movement.css is out of the way.

[kkirsche]

@alber70g It doesn't read that way to me — this would put them in a separate files block. main would retain source files only is my understanding of the OP's post.

[HeroBart]

So we can expect half a year of wiredep not working and dependency mayhem... Nice.

[alber70g]

files is an array of all the files required to use the package. Filepaths may include globs.

files differs from main in that main only lists single files per filetype, whereas files should include every file.

We recommend listing source files in files, rather than compiled distribution files.

The second line implies that we should keep only ONE file per filetype in main
The last line implies that we should put into files what is now in main in a sense that .less (or scss) is not allowed to be in main (which is the case, by looking at the current spec).

[nmalaguti]

Who is files meant to be consumed by? Any sufficiently advanced packer should be able to parse the assets listed in main and resolve all other dependencies (scripts, styles, images, fonts, etc.).

The problem is that naive dependency resolvers like wiredep don't have much file specific knowledge, they just want to resolve paths and group assets by extension. Depending on the file extension, they are injected into html or returned in an object (organized by extension/dependency) for further tools to consume.

There are several kinds of information I would find useful from a package (property names aren't important here):

• main: entry points for various file types that can be directly referenced from a browser. Not all file types support imports (e.g. ES5) and therefore concat + minification may be desirable for some.
• metadata: e.g. .d.ts or .less files with typing or variable information, but that compile down to nothing. Useful when doing development.
• sources: a list of source entry point files in the component that are needed for compilation (if no sources use imports/exports, then all source files should be listed). Don't list intermediate representations (e.g. ts -> js -> min.js, the js files should not be listed).
• assets: a list of static assets that are referenced by various other files (e.g. fonts, images, etc.) that should be copied into production so that unneeded files (sources, metadata) can be filtered out.

It may also be useful to offer different collections of files in the package that can be combined together with main as needed by the consumer (think various sizes, legacy support, languages):

{
  "main": [
    "js/motion-base.js"
  ],
  "options": {
    "lite": [
      "js/motion-lite.js"
    ],
    "full": [
      "js/motion-full.js"
    ],
    "huge": [
      "js/motion-huge.js"
    ],
    "legacy": [
      "js/motion-legacy.js"
    ]
  }
}

There would likely need to be some way for the consumer to specify which option(s) they want to be combined with main.

I think this gives enough information for a consumer to take the bower.json and pipe the various file lists into their tools of choice.

• metadata is used by the consumer to build their own projects/do development
• sources can go to a compiler (and then main should be ignored as you don't want to include stuff twice)
• main + options are injected as paths into html (or whatever you want to do with file paths)
• assets are retained when shipping to production so all unneccessary files can be stripped out (I already minified all js and therefore don't need to ship it)

[cvrebert]

• Re: Proposal 1: files flat list:
  • Couldn't tools just do ls -R . and then subtract out the ignore entries and get the same result? If yes, I don't see the point; Bower or the tools could just generate the list programmatically at runtime instead.
• Re: Proposal 2: asset-type object:
  • Better, but I think it would be good to tie precompiled CSS to the preprocessor file that it originated from, which this proposal doesn't do.
  • Example/strawman:

"files": {
  "styles": {
    "dist/css/bootstrap.css": "less/bootstrap.less",
    "dist/css/bootstrap-theme.css": "less/theme.less"
  }
}

Other issues:

• What about bootstrap.min.css.map?
  • Can we assume the <foo>.map naming convention and not explicitly list these and just have tools check for them?
• What about bootstrap.min.css and bootstrap.min.js?
  • Can we similarly assume the <foo>.min.<bar> naming convention?

[ezraroi]

I think proposal 1 is better, build tools will easily adopt it I believe