You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

beam.xml 15KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424
  1. <section xmlns="http://docbook.org/ns/docbook"
  2. xmlns:xlink="http://www.w3.org/1999/xlink"
  3. xml:id="sec-beam">
  4. <title>BEAM Languages (Erlang, Elixir &amp; LFE)</title>
  5. <section xml:id="beam-introduction">
  6. <title>Introduction</title>
  7. <para>
  8. In this document and related Nix expressions, we use the term, <emphasis>BEAM</emphasis>, to describe the environment. BEAM is the name of the Erlang Virtual Machine and, as far as we're concerned, from a packaging perspective, all languages that run on the BEAM are interchangeable. That which varies, like the build system, is transparent to users of any given BEAM package, so we make no distinction.
  9. </para>
  10. </section>
  11. <section xml:id="beam-structure">
  12. <title>Structure</title>
  13. <para>
  14. All BEAM-related expressions are available via the top-level <literal>beam</literal> attribute, which includes:
  15. </para>
  16. <itemizedlist>
  17. <listitem>
  18. <para>
  19. <literal>interpreters</literal>: a set of compilers running on the BEAM, including multiple Erlang/OTP versions (<literal>beam.interpreters.erlangR19</literal>, etc), Elixir (<literal>beam.interpreters.elixir</literal>) and LFE (<literal>beam.interpreters.lfe</literal>).
  20. </para>
  21. </listitem>
  22. <listitem>
  23. <para>
  24. <literal>packages</literal>: a set of package sets, each compiled with a specific Erlang/OTP version, e.g. <literal>beam.packages.erlangR19</literal>.
  25. </para>
  26. </listitem>
  27. </itemizedlist>
  28. <para>
  29. The default Erlang compiler, defined by <literal>beam.interpreters.erlang</literal>, is aliased as <literal>erlang</literal>. The default BEAM package set is defined by <literal>beam.packages.erlang</literal> and aliased at the top level as <literal>beamPackages</literal>.
  30. </para>
  31. <para>
  32. To create a package set built with a custom Erlang version, use the lambda, <literal>beam.packagesWith</literal>, which accepts an Erlang/OTP derivation and produces a package set similar to <literal>beam.packages.erlang</literal>.
  33. </para>
  34. <para>
  35. Many Erlang/OTP distributions available in <literal>beam.interpreters</literal> have versions with ODBC and/or Java enabled. For example, there's <literal>beam.interpreters.erlangR19_odbc_javac</literal>, which corresponds to <literal>beam.interpreters.erlangR19</literal>.
  36. </para>
  37. <para xml:id="erlang-call-package">
  38. We also provide the lambda, <literal>beam.packages.erlang.callPackage</literal>, which simplifies writing BEAM package definitions by injecting all packages from <literal>beam.packages.erlang</literal> into the top-level context.
  39. </para>
  40. </section>
  41. <section xml:id="build-tools">
  42. <title>Build Tools</title>
  43. <section xml:id="build-tools-rebar3">
  44. <title>Rebar3</title>
  45. <para>
  46. We provide a version of Rebar3, which is the normal, unmodified Rebar3, under <literal>rebar3</literal>. We also provide a helper to fetch Rebar3 dependencies from a lockfile under <literal>fetchRebar3Deps</literal>.
  47. </para>
  48. </section>
  49. <section xml:id="build-tools-other">
  50. <title>Mix &amp; Erlang.mk</title>
  51. <para>
  52. Both Mix and Erlang.mk work exactly as expected. There is a bootstrap process that needs to be run for both, however, which is supported by the <literal>buildMix</literal> and <literal>buildErlangMk</literal> derivations, respectively.
  53. </para>
  54. </section>
  55. </section>
  56. <section xml:id="how-to-install-beam-packages">
  57. <title>How to Install BEAM Packages</title>
  58. <para>
  59. BEAM packages are not registered at the top level, simply because they are not relevant to the vast majority of Nix users. They are installable using the <literal>beam.packages.erlang</literal> attribute set (aliased as <literal>beamPackages</literal>), which points to packages built by the default Erlang/OTP version in Nixpkgs, as defined by <literal>beam.interpreters.erlang</literal>. To list the available packages in <literal>beamPackages</literal>, use the following command:
  60. </para>
  61. <screen>
  62. <prompt>$ </prompt>nix-env -f &quot;&lt;nixpkgs&gt;&quot; -qaP -A beamPackages
  63. beamPackages.esqlite esqlite-0.2.1
  64. beamPackages.goldrush goldrush-0.1.7
  65. beamPackages.ibrowse ibrowse-4.2.2
  66. beamPackages.jiffy jiffy-0.14.5
  67. beamPackages.lager lager-3.0.2
  68. beamPackages.meck meck-0.8.3
  69. beamPackages.rebar3-pc pc-1.1.0
  70. </screen>
  71. <para>
  72. To install any of those packages into your profile, refer to them by their attribute path (first column):
  73. </para>
  74. <screen>
  75. <prompt>$ </prompt>nix-env -f &quot;&lt;nixpkgs&gt;&quot; -iA beamPackages.ibrowse
  76. </screen>
  77. <para>
  78. The attribute path of any BEAM package corresponds to the name of that particular package in <link xlink:href="https://hex.pm">Hex</link> or its OTP Application/Release name.
  79. </para>
  80. </section>
  81. <section xml:id="packaging-beam-applications">
  82. <title>Packaging BEAM Applications</title>
  83. <section xml:id="packaging-erlang-applications">
  84. <title>Erlang Applications</title>
  85. <section xml:id="rebar3-packages">
  86. <title>Rebar3 Packages</title>
  87. <para>
  88. The Nix function, <literal>buildRebar3</literal>, defined in <literal>beam.packages.erlang.buildRebar3</literal> and aliased at the top level, can be used to build a derivation that understands how to build a Rebar3 project. For example, we can build <link
  89. xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link> as follows:
  90. </para>
  91. <programlisting>
  92. { stdenv, fetchFromGitHub, buildRebar3, ibrowse, jsx, erlware_commons }:
  93. buildRebar3 rec {
  94. name = "hex2nix";
  95. version = "0.0.1";
  96. src = fetchFromGitHub {
  97. owner = "ericbmerritt";
  98. repo = "hex2nix";
  99. rev = "${version}";
  100. sha256 = "1w7xjidz1l5yjmhlplfx7kphmnpvqm67w99hd2m7kdixwdxq0zqg";
  101. };
  102. beamDeps = [ ibrowse jsx erlware_commons ];
  103. }
  104. </programlisting>
  105. <para>
  106. Such derivations are callable with <literal>beam.packages.erlang.callPackage</literal> (see <xref
  107. linkend="erlang-call-package"/>). To call this package using the normal <literal>callPackage</literal>, refer to dependency packages via <literal>beamPackages</literal>, e.g. <literal>beamPackages.ibrowse</literal>.
  108. </para>
  109. <para>
  110. Notably, <literal>buildRebar3</literal> includes <literal>beamDeps</literal>, while <literal>stdenv.mkDerivation</literal> does not. BEAM dependencies added there will be correctly handled by the system.
  111. </para>
  112. <para>
  113. If a package needs to compile native code via Rebar3's port compilation mechanism, add <literal>compilePort = true;</literal> to the derivation.
  114. </para>
  115. </section>
  116. <section xml:id="erlang-mk-packages">
  117. <title>Erlang.mk Packages</title>
  118. <para>
  119. Erlang.mk functions similarly to Rebar3, except we use <literal>buildErlangMk</literal> instead of <literal>buildRebar3</literal>.
  120. </para>
  121. <programlisting>
  122. { buildErlangMk, fetchHex, cowlib, ranch }:
  123. buildErlangMk {
  124. name = "cowboy";
  125. version = "1.0.4";
  126. src = fetchHex {
  127. pkg = "cowboy";
  128. version = "1.0.4";
  129. sha256 = "6a0edee96885fae3a8dd0ac1f333538a42e807db638a9453064ccfdaa6b9fdac";
  130. };
  131. beamDeps = [ cowlib ranch ];
  132. meta = {
  133. description = ''
  134. Small, fast, modular HTTP server written in Erlang
  135. '';
  136. license = stdenv.lib.licenses.isc;
  137. homepage = https://github.com/ninenines/cowboy;
  138. };
  139. }
  140. </programlisting>
  141. </section>
  142. <section xml:id="mix-packages">
  143. <title>Mix Packages</title>
  144. <para>
  145. Mix functions similarly to Rebar3, except we use <literal>buildMix</literal> instead of <literal>buildRebar3</literal>.
  146. </para>
  147. <programlisting>
  148. { buildMix, fetchHex, plug, absinthe }:
  149. buildMix {
  150. name = "absinthe_plug";
  151. version = "1.0.0";
  152. src = fetchHex {
  153. pkg = "absinthe_plug";
  154. version = "1.0.0";
  155. sha256 = "08459823fe1fd4f0325a8bf0c937a4520583a5a26d73b193040ab30a1dfc0b33";
  156. };
  157. beamDeps = [ plug absinthe ];
  158. meta = {
  159. description = ''
  160. A plug for Absinthe, an experimental GraphQL toolkit
  161. '';
  162. license = stdenv.lib.licenses.bsd3;
  163. homepage = https://github.com/CargoSense/absinthe_plug;
  164. };
  165. }
  166. </programlisting>
  167. <para>
  168. Alternatively, we can use <literal>buildHex</literal> as a shortcut:
  169. </para>
  170. <programlisting>
  171. { buildHex, buildMix, plug, absinthe }:
  172. buildHex {
  173. name = "absinthe_plug";
  174. version = "1.0.0";
  175. sha256 = "08459823fe1fd4f0325a8bf0c937a4520583a5a26d73b193040ab30a1dfc0b33";
  176. builder = buildMix;
  177. beamDeps = [ plug absinthe ];
  178. meta = {
  179. description = ''
  180. A plug for Absinthe, an experimental GraphQL toolkit
  181. '';
  182. license = stdenv.lib.licenses.bsd3;
  183. homepage = https://github.com/CargoSense/absinthe_plug;
  184. };
  185. }
  186. </programlisting>
  187. </section>
  188. </section>
  189. </section>
  190. <section xml:id="how-to-develop">
  191. <title>How to Develop</title>
  192. <section xml:id="accessing-an-environment">
  193. <title>Accessing an Environment</title>
  194. <para>
  195. Often, we simply want to access a valid environment that contains a specific package and its dependencies. We can accomplish that with the <literal>env</literal> attribute of a derivation. For example, let's say we want to access an Erlang REPL with <literal>ibrowse</literal> loaded up. We could do the following:
  196. </para>
  197. <screen>
  198. <prompt>$ </prompt><userinput>nix-shell -A beamPackages.ibrowse.env --run "erl"</userinput>
  199. <computeroutput>Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
  200. Eshell V7.0 (abort with ^G)</computeroutput>
  201. <prompt>1> </prompt><userinput>m(ibrowse).</userinput>
  202. <computeroutput>Module: ibrowse
  203. MD5: 3b3e0137d0cbb28070146978a3392945
  204. Compiled: January 10 2016, 23:34
  205. Object file: /nix/store/g1rlf65rdgjs4abbyj4grp37ry7ywivj-ibrowse-4.2.2/lib/erlang/lib/ibrowse-4.2.2/ebin/ibrowse.beam
  206. Compiler options: [{outdir,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/ebin"},
  207. debug_info,debug_info,nowarn_shadow_vars,
  208. warn_unused_import,warn_unused_vars,warnings_as_errors,
  209. {i,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/include"}]
  210. Exports:
  211. add_config/1 send_req_direct/7
  212. all_trace_off/0 set_dest/3
  213. code_change/3 set_max_attempts/3
  214. get_config_value/1 set_max_pipeline_size/3
  215. get_config_value/2 set_max_sessions/3
  216. get_metrics/0 show_dest_status/0
  217. get_metrics/2 show_dest_status/1
  218. handle_call/3 show_dest_status/2
  219. handle_cast/2 spawn_link_worker_process/1
  220. handle_info/2 spawn_link_worker_process/2
  221. init/1 spawn_worker_process/1
  222. module_info/0 spawn_worker_process/2
  223. module_info/1 start/0
  224. rescan_config/0 start_link/0
  225. rescan_config/1 stop/0
  226. send_req/3 stop_worker_process/1
  227. send_req/4 stream_close/1
  228. send_req/5 stream_next/1
  229. send_req/6 terminate/2
  230. send_req_direct/4 trace_off/0
  231. send_req_direct/5 trace_off/2
  232. send_req_direct/6 trace_on/0
  233. trace_on/2
  234. ok</computeroutput>
  235. <prompt>2></prompt>
  236. </screen>
  237. <para>
  238. Notice the <literal>-A beamPackages.ibrowse.env</literal>. That is the key to this functionality.
  239. </para>
  240. </section>
  241. <section xml:id="creating-a-shell">
  242. <title>Creating a Shell</title>
  243. <para>
  244. Getting access to an environment often isn't enough to do real development. Usually, we need to create a <literal>shell.nix</literal> file and do our development inside of the environment specified therein. This file looks a lot like the packaging described above, except that <literal>src</literal> points to the project root and we call the package directly.
  245. </para>
  246. <programlisting>
  247. { pkgs ? import &quot;&lt;nixpkgs&quot;&gt; {} }:
  248. with pkgs;
  249. let
  250. f = { buildRebar3, ibrowse, jsx, erlware_commons }:
  251. buildRebar3 {
  252. name = "hex2nix";
  253. version = "0.1.0";
  254. src = ./.;
  255. beamDeps = [ ibrowse jsx erlware_commons ];
  256. };
  257. drv = beamPackages.callPackage f {};
  258. in
  259. drv
  260. </programlisting>
  261. <section xml:id="building-in-a-shell">
  262. <title>Building in a Shell (for Mix Projects)</title>
  263. <para>
  264. We can leverage the support of the derivation, irrespective of the build derivation, by calling the commands themselves.
  265. </para>
  266. <programlisting>
  267. # =============================================================================
  268. # Variables
  269. # =============================================================================
  270. NIX_TEMPLATES := "$(CURDIR)/nix-templates"
  271. TARGET := "$(PREFIX)"
  272. PROJECT_NAME := thorndyke
  273. NIXPKGS=../nixpkgs
  274. NIX_PATH=nixpkgs=$(NIXPKGS)
  275. NIX_SHELL=nix-shell -I "$(NIX_PATH)" --pure
  276. # =============================================================================
  277. # Rules
  278. # =============================================================================
  279. .PHONY= all test clean repl shell build test analyze configure install \
  280. test-nix-install publish plt analyze
  281. all: build
  282. guard-%:
  283. @ if [ "${${*}}" == "" ]; then \
  284. echo "Environment variable $* not set"; \
  285. exit 1; \
  286. fi
  287. clean:
  288. rm -rf _build
  289. rm -rf .cache
  290. repl:
  291. $(NIX_SHELL) --run "iex -pa './_build/prod/lib/*/ebin'"
  292. shell:
  293. $(NIX_SHELL)
  294. configure:
  295. $(NIX_SHELL) --command 'eval "$$configurePhase"'
  296. build: configure
  297. $(NIX_SHELL) --command 'eval "$$buildPhase"'
  298. install:
  299. $(NIX_SHELL) --command 'eval "$$installPhase"'
  300. test:
  301. $(NIX_SHELL) --command 'mix test --no-start --no-deps-check'
  302. plt:
  303. $(NIX_SHELL) --run "mix dialyzer.plt --no-deps-check"
  304. analyze: build plt
  305. $(NIX_SHELL) --run "mix dialyzer --no-compile"
  306. </programlisting>
  307. <para>
  308. Using a <literal>shell.nix</literal> as described (see <xref
  309. linkend="creating-a-shell"/>) should just work. Aside from <literal>test</literal>, <literal>plt</literal>, and <literal>analyze</literal>, the Make targets work just fine for all of the build derivations.
  310. </para>
  311. </section>
  312. </section>
  313. </section>
  314. <section xml:id="generating-packages-from-hex-with-hex2nix">
  315. <title>Generating Packages from Hex with <literal>hex2nix</literal></title>
  316. <para>
  317. Updating the <link xlink:href="https://hex.pm">Hex</link> package set requires <link
  318. xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link>. Given the path to the Erlang modules (usually <literal>pkgs/development/erlang-modules</literal>), it will dump a file called <literal>hex-packages.nix</literal>, containing all the packages that use a recognized build system in <link
  319. xlink:href="https://hex.pm">Hex</link>. It can't be determined, however, whether every package is buildable.
  320. </para>
  321. <para>
  322. To make life easier for our users, try to build every <link
  323. xlink:href="https://hex.pm">Hex</link> package and remove those that fail. To do that, simply run the following command in the root of your <literal>nixpkgs</literal> repository:
  324. </para>
  325. <screen>
  326. <prompt>$ </prompt>nix-build -A beamPackages
  327. </screen>
  328. <para>
  329. That will attempt to build every package in <literal>beamPackages</literal>. Then manually remove those that fail. Hopefully, someone will improve <link
  330. xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link> in the future to automate the process.
  331. </para>
  332. </section>
  333. </section>