TipsAndTricks/NPM - Revision history

Michael Opdenacker: /* Yocto 2.1.1 (krogoth) Restrictions */ Remove this obsolete section

2022-06-22T13:27:09Z

Yocto 2.1.1 (krogoth) Restrictions: Remove this obsolete section

@@ Line 43: / Line 43: @@
+            "
 You can see that the main module is taken from git repo and dependents obtained from registry. Other than that, recipe is fairly similar and can be built and deployed as above.
 == Yocto 3.1 (dunfell) update ==

Michael Opdenacker: /* Creating NPM Packages */ Replace some content by a link to the manual

2022-06-22T13:25:58Z

Creating NPM Packages: Replace some content by a link to the manual

@@ Line 30: / Line 30: @@
 == Creating NPM Packages ==
-There are a few requirements and caveats
+Documented on [https://docs.yoctoproject.org/dev-manual/common-tasks.html#creating-node-package-manager-npm-packages the Yocto Project manual]
 === NPM Projects in Development ===

Albert Babí Oller: Add information on how to migrate npm recipes to dunfell release

2021-10-14T12:21:05Z

Add information on how to migrate npm recipes to dunfell release

← Older revision		Revision as of 12:21, 14 October 2021
Line 148:		Line 148:
	To quickly test the module, copy to target		To quickly test the module, copy to target
	scp -r tmp/work/${TARGET_SYS}/cute-files/1.0-r0/package/usr/lib/node_modules/cute-files/ root@192.168.7.2:/usr/lib/node_modules		scp -r tmp/work/${TARGET_SYS}/cute-files/1.0-r0/package/usr/lib/node_modules/cute-files/ root@192.168.7.2:/usr/lib/node_modules


			== Yocto 3.1 (dunfell) update ==

			The [https://docs.yoctoproject.org/3.2.3/ref-manual/migration-3.1.html#npm-fetcher-changes npm fetch system] has been [https://blog.savoirfairelinux.com/en-ca/2020/dealing-with-angular-npm-dependencies-on-embedded-systems/ refactored] for the [https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#creating-node-package-manager-npm-packages dunfell release].

			Importing a recipe that uses npm from zeus to dunfell it is not an straightforward process. Syntax and fetch has changed, so you might encounter errors like the following:

			<pre>
			ERROR: cute-files-1.0.2-r0 do_fetch: URL: 'npm://registry.npmjs.org/;name=cute-files;version=1.0.2' is missing the required parameter 'Parameter 'package' required'
			</pre>

			In this case the error is related to the registry syntax. When using the registry url (from devtool or in your recipe) from dunfell release, the package name is specified by the "package" parameter instead of "name". Other errors might occur, so importing npm recipes to dunfell release might require a migration process.

			=== Migrating NPM recipes ===

			1. Regenerate the shrinkwrap file

			As described above, the ''devtool add'' offers the possibility to create recipes and shrinkwrap files for npm applications (either from registry, git, local sources...).

			You can create an updated shrinkwrap for your application with:

			<pre>
			devtool add <app>
			</pre>

			where ''<app>'' refers to the npm registry, git repository or local sources to your application.

			Once the command has been successfully executed you can overwrite your previous shrinkwrap with the content of ''build/workspace/recipes/<app>/<app>/npm-shrinkwrap.json''.

			2. Clean workspace

			In order to avoid duplicated recipes you should remove the newly generated one:

			<pre>
			rm -rf build/workspace/recipes/<app> build/workspace/appends/<app> build/workspace/sources/<app>
			</pre>

			3. Adapt old recipe

			* Update ''SRC_URI'' variable: A common use case for npm developers is to integrate a non-registered application that has dependencies to fetch from the registry. As described on the [https://www.yoctoproject.org/docs/3.0.4/mega-manual/mega-manual.html#creating-node-package-manager-npm-packages documentation], until zeus release, this recipe would describe the sources to fetch with the following syntax:

			<pre>
			SRC_URI = "git://github.com/martinaglv/cute-files.git;protocol=https \
			npm://registry.npmjs.org;name=commander;version=2.9.0;subdir=node_modules/commander \
			npm://registry.npmjs.org;name=express;version=4.14.0;subdir=node_modules/express \
			npm://registry.npmjs.org;name=content-disposition;version=0.3.0;subdir=node_modules/content-disposition \
			"
			</pre>

			where the first line represents the repository for the main package and the other three the registry dependencies.

			However, with the refactoring made on the [https://docs.yoctoproject.org/3.2.3/ref-manual/migration-3.1.html#npm-fetcher-changes dunfell release], the fetcher is capable of getting all the dependencies from a single shrinkwrap file (which is very convenient for large applications). So the ''SRC_URI'' of your recipe can be left as:

			<pre>
			SRC_URI = " \
			git://github.com/tutorialzine/cute-files.git;protocol=https \
			npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \
			"
			</pre>

			Note that the syntax uses ''npmsw://'' instead of ''npm://''.

			* Remove obsolete variables: Another change that might be necessary is to remove the obsolete variables ''NPM_SHRINKWRAP'' and ''NPM_LOCKDOWN'' from your recipe.

			<pre>
			NPM_SHRINKWRAP := "${THISDIR}/${PN}/npm-shrinkwrap.json"
			NPM_LOCKDOWN := "${THISDIR}/${PN}/package-lock.json"
			</pre>

			* Remove ''package-lock.json'': The file ''package-lock.json'' can also be removed from your recipe folder.

Eduardo Abinader: /* NPM Projects in Development */

2018-06-26T06:28:10Z

NPM Projects in Development

← Older revision		Revision as of 06:28, 26 June 2018
Line 83:		Line 83:

	=== NPM Projects in Development ===		=== NPM Projects in Development ===
	Although it is useful to package modules already in the registry, adding node.js projects in development is a more common developer use case. The is very similar to he "registry" approach but instead we give devtool a URL to the source code. To create the cute-files example above from source code would we done as follows		Although it is useful to package modules already in the registry, adding node.js projects in development is a more common developer use case. The is very similar to the "registry" approach, but instead we give devtool a URL to the source code. To create the cute-files example above from source code we would have done as follows:
	devtool add https://github.com/martinaglv/cute-files.git		devtool add https://github.com/martinaglv/cute-files.git
	If you look at the ~~genenrated~~ recipe, it will be very similar, but the SRC_URI will look like this		If you look at the generated recipe, it will be very similar, but the SRC_URI will look like this:
	SRC_URI = "git://github.com/martinaglv/cute-files.git;protocol=https \		SRC_URI = "git://github.com/martinaglv/cute-files.git;protocol=https \
	npm://registry.npmjs.org;name=commander;version=2.9.0;subdir=node_modules/commander \		npm://registry.npmjs.org;name=commander;version=2.9.0;subdir=node_modules/commander \

Eduardo Abinader: /* Registry Modules */

2018-06-26T06:25:20Z

Registry Modules

← Older revision		Revision as of 06:25, 26 June 2018
Line 73:		Line 73:
	LICENSE_${PN} = "MIT"		LICENSE_${PN} = "MIT"
	</pre>		</pre>
	You can ~~run~~ run <tt>devtool build cute-files</tt> to build package. Remember that <tt>nodejs</tt> must be installed on the target before deploying. Assuming IP address of target is 192.168.7.2, deploy as follows.		You can run <tt>devtool build cute-files</tt> to build package. Remember that <tt>nodejs</tt> must be installed on the target before deploying. Assuming IP address of target is 192.168.7.2, deploy as follows:
	devtool deploy-target -s cute-files root@192.168.7.2		devtool deploy-target -s cute-files root@192.168.7.2
	Once installed on target, test app as follows (note that you can't simply run cute-files as if you'd done an 'npm install'. This is a [https://bugzilla.yoctoproject.org/show_bug.cgi?id=10460 known issue]).		Once installed on the target, test app as follows (note that you can't simply run cute-files as if you'd done an 'npm install'. This is a [https://bugzilla.yoctoproject.org/show_bug.cgi?id=10460 known issue]).
	cd /usr/lib/node_modules/cute-files		cd /usr/lib/node_modules/cute-files
	node cute-files.js		node cute-files.js
	Then on a browser go to http://192.168.7.2:3000 and you'll see the following		Then, on a browser go to http://192.168.7.2:3000 and you'll see the following
	:[[File:npm-cute-files-screenshot.png\|640px]]		:[[File:npm-cute-files-screenshot.png\|640px]]
	Recipe can be found in workspace/recipes/cute-files and can be added to layer of your choice.		Recipe can be found in workspace/recipes/cute-files and can be added to layer of your choice.

Eduardo Abinader: /* Registry Modules */

2018-06-26T06:22:20Z

Registry Modules

← Older revision		Revision as of 06:22, 26 June 2018
Line 45:		Line 45:
	Under the hood devtool, runs '''recipetool create''' with the same fetch uri. Recipetool downloads each dependency, capturing licence details where possible and generates a recipe file. The recipe file is fairly simple but will contain every license that recipetool has found and include it in the <tt>LIC_FILES_CHKSUM</tt>. Many node modules have unclear licensing so you'll see "unknown" in the LICENSE field. Have a look at the modules not listed.		Under the hood devtool, runs '''recipetool create''' with the same fetch uri. Recipetool downloads each dependency, capturing licence details where possible and generates a recipe file. The recipe file is fairly simple but will contain every license that recipetool has found and include it in the <tt>LIC_FILES_CHKSUM</tt>. Many node modules have unclear licensing so you'll see "unknown" in the LICENSE field. Have a look at the modules not listed.

	Recipetool will also create ''shrinkwrap'' and ''lockdown'' files for your recipe. Shrinkwrap files capture the version of all dependent modules. Many packages don't provide this so we create one on the fly. You can replace it with your own file by setting the <tt>NPM_SHRINKWRAP</tt> variable. Lockdown files contain the checksum for each module to check your users will download the same files when building with your recipe. This ensure that dependencies have not been changed and that your NPM registry is still handing out the same file. Note also that a package is created for each sub module. This may seem like overkill but it's the only practical way to have the licenses for all of the dependencies represented in the license manifest ~~for~~ the image.		Recipetool will also create ''shrinkwrap'' and ''lockdown'' files for your recipe. Shrinkwrap files capture the version of all dependent modules. Many packages don't provide this so we create one on the fly. You can replace it with your own file by setting the <tt>NPM_SHRINKWRAP</tt> variable. Lockdown files contain the checksum for each module to check if your users will download the same files when building with your recipe. This ensure that dependencies have not been changed and that your NPM registry is still handing out the same file. Note also that a package is created for each sub module. This may seem like overkill but it's the only practical way to have the licenses for all of the dependencies represented in the license manifest of the image.

	You can ~~tool~~ a look at the generate recipe with		You can take a look at the generate recipe with
	devtool edit-recipe cute-files		devtool edit-recipe cute-files
	There are three key points to note		There are three key points to note

Eduardo Abinader: /* Registry Modules */

2018-06-26T06:20:10Z

Registry Modules

← Older revision		Revision as of 06:20, 26 June 2018
Line 43:		Line 43:
	In this example, we'll use the cute-files module that is file browser web app. First we'll use the registry approach. Note that you need to know the module version.		In this example, we'll use the cute-files module that is file browser web app. First we'll use the registry approach. Note that you need to know the module version.
	devtool add "npm://registry.npmjs.org;name=cute-files;version=1.0.2"		devtool add "npm://registry.npmjs.org;name=cute-files;version=1.0.2"
	Under the hood devtool runs '''recipetool create''' with the same fetch uri. Recipetool downloads each dependency, capturing licence details where possible and generates a recipe file. The recipe file is fairly simple but will contain every license that recipetool has found and include it in the <tt>LIC_FILES_CHKSUM</tt>. Many node modules have unclear licensing so you'll see "unknown" in the LICENSE field. Have a look at the modules not listed.		Under the hood devtool, runs '''recipetool create''' with the same fetch uri. Recipetool downloads each dependency, capturing licence details where possible and generates a recipe file. The recipe file is fairly simple but will contain every license that recipetool has found and include it in the <tt>LIC_FILES_CHKSUM</tt>. Many node modules have unclear licensing so you'll see "unknown" in the LICENSE field. Have a look at the modules not listed.

	Recipetool will also create ''shrinkwrap'' and ''lockdown'' files for your recipe. Shrinkwrap files capture the version of all dependent modules. Many packages don't provide this so we create one on the fly. You can replace it with your own file by setting the <tt>NPM_SHRINKWRAP</tt> variable. Lockdown files contain the checksum for each module to check your users will download the same files when building with your recipe. This ensure that dependencies have not been changed and that your NPM registry is still handing out the same file. Note also that a package is created for each sub module. This may seem like overkill but it's the only practical way to have the licenses for all of the dependencies represented in the license manifest for the image.		Recipetool will also create ''shrinkwrap'' and ''lockdown'' files for your recipe. Shrinkwrap files capture the version of all dependent modules. Many packages don't provide this so we create one on the fly. You can replace it with your own file by setting the <tt>NPM_SHRINKWRAP</tt> variable. Lockdown files contain the checksum for each module to check your users will download the same files when building with your recipe. This ensure that dependencies have not been changed and that your NPM registry is still handing out the same file. Note also that a package is created for each sub module. This may seem like overkill but it's the only practical way to have the licenses for all of the dependencies represented in the license manifest for the image.

Eduardo Abinader: /* Introduction and Caveats */

2018-06-26T06:19:28Z

Introduction and Caveats

← Older revision		Revision as of 06:19, 26 June 2018
Line 38:		Line 38:
	* Devtool cannot detect native libraries in module dependencies, you'll need to manually add packages to recipe		* Devtool cannot detect native libraries in module dependencies, you'll need to manually add packages to recipe
	* At deployment time devtool cannot determine which dependent packages are missing on the target (e.g. the node runtime, <tt>nodejs</tt>), so you have to do that manually.		* At deployment time devtool cannot determine which dependent packages are missing on the target (e.g. the node runtime, <tt>nodejs</tt>), so you have to do that manually.
	* Although npm may not be needed to run your node package, it is useful to have on the target. Package name is <tt>nodejs-npm</tt>		* Although npm may not be needed to run your node package, it is useful to have on the target. Package name is <tt>nodejs-npm</tt>.

	=== Registry Modules ===		=== Registry Modules ===

Eduardo Abinader: /* Introduction and Caveats */

2018-06-26T06:16:36Z

Introduction and Caveats

← Older revision		Revision as of 06:16, 26 June 2018
Line 36:		Line 36:
	* You need to be familiar with [http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#using-devtool-in-your-workflow devtool] to get the best out of this article		* You need to be familiar with [http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#using-devtool-in-your-workflow devtool] to get the best out of this article
	* The npm host tools need the native nodejs-npm package which is part of meta-oe. You must get the meta-openembedded layer from git://git.openembedded.org/meta-openembedded and add /path/to/meta-openembedded/meta-oe to conf/bblayers.conf.		* The npm host tools need the native nodejs-npm package which is part of meta-oe. You must get the meta-openembedded layer from git://git.openembedded.org/meta-openembedded and add /path/to/meta-openembedded/meta-oe to conf/bblayers.conf.
	* Devtool cannot detect native libraries in module dependencies, ~~you~~ you'll need to manually add packages to recipe		* Devtool cannot detect native libraries in module dependencies, you'll need to manually add packages to recipe
	* At deployment time devtool cannot determine which dependent packages are missing on the target (e.g. the node runtime, <tt>nodejs</tt>), so you have to do that manually.		* At deployment time devtool cannot determine which dependent packages are missing on the target (e.g. the node runtime, <tt>nodejs</tt>), so you have to do that manually.
	* Although npm may not be needed to run your node package, it is useful to have on the target. Package name is <tt>nodejs-npm</tt>		* Although npm may not be needed to run your node package, it is useful to have on the target. Package name is <tt>nodejs-npm</tt>

Eduardo Abinader: /* Introduction and Caveats */

2018-06-26T06:14:51Z

Introduction and Caveats

← Older revision		Revision as of 06:14, 26 June 2018
Line 31:		Line 31:
	== Creating NPM Packages ==		== Creating NPM Packages ==
	=== Introduction and Caveats ===		=== Introduction and Caveats ===
	Although npm recipes can be created manually, using [http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#using-devtool-in-your-workflow '''devtool'''] ~~make~~ the job much easier. There are two options for creating recipe, using the NPM registry and NPM project source code. The registry approach is slightly simpler, but you're more likely to use the project approach as you don't have to publish your module in the registry.		Although npm recipes can be created manually, using [http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#using-devtool-in-your-workflow '''devtool'''] makes the job much easier. There are two options for creating recipe, using the NPM registry and NPM project source code. The registry approach is slightly simpler, but you're more likely to use the project approach as you don't have to publish your module in the registry.

	There are a few requirements and caveats		There are a few requirements and caveats