While benchmarking the example from #769 I saw that a significant
amount of time was spent benchmarking record literals. When I looked
at the code more closely I saw that the first key in the record literal
was being type-checked twice (once to figure out the record's associated
type-checking constant and once as part of the `process` loop).
This change fixes that, which speeds up interpretation of the large
example by 9%:
Before:
```
time 18.13 s (18.11 s .. 18.16 s)
1.000 R² (1.000 R² .. 1.000 R²)
mean 18.09 s (18.07 s .. 18.11 s)
std dev 21.92 ms (10.66 ms .. 29.76 ms)
variance introduced by outliers: 19% (moderately inflated)
```
After:
```
time 16.53 s (16.49 s .. 16.60 s)
1.000 R² (1.000 R² .. 1.000 R²)
mean 16.59 s (16.56 s .. 16.64 s)
std dev 43.65 ms (6.227 ms .. 56.35 ms)
variance introduced by outliers: 19% (moderately inflated)
```
Related to: https://github.com/dhall-lang/dhall-haskell/issues/1039
We'll probably never see indices that exceed the space of an `Int` and
the interpreter would probably not be able to handle 9223372036854775807
nested binders anyway.
This reduces the runtime of the `deep-nested-large-record` benchmark by about 50%.
Note that previously, contrary to its name and documentation, this function traversed a Dhall.Map in insertion order due to its use of Dhall.Map.toList! With this change, the traversal is changed to ascending key order.
Also:
- Fix the deep-nested-large-record benchmark
- Remove the map-operations benchmark: This benchmark actually reports a ~20% loss of performance for the unorderedTraverseWithKey_ change. Since we eventually care about dhall's performance we it's better not to be mislead by a questionable micro-benchmark.
This separates the source from the line numbers using vertical bars
instead of colons. The main reason for this is to more clearly
delimit the two and to also prevent the old colon from being confused
as a type annotation.
Fixes https://github.com/dhall-lang/dhall-haskell/issues/1025
There were two error constructors related to invalid field types:
* `InvalidField`: for field types whose types were invalid type-checking
constants
* `InvalidFieldType`: for all other invalid field types
As @sjakobi noted, there are no invalid field types for the former
category, so we can remove that category of error entirely.
Note that `InvalidField` was still being used in a few places, but each
of those uses should actually have been using `InvalidFieldType`
instead, which this change also fixes.
This improves the error message when users attempt to apply `merge` to only
one argument, as suggested by:
https://github.com/dhall-lang/dhall-lang/issues/106#issuecomment-503523358
For example:
```
$ dhall <<< 'merge { Foo = True }'
Error: Invalid input
(stdin):2:1:
|
2 | <empty line>
| ^
unexpected end of input
expecting '.', second argument to ❰merge❱, or whitespace
```
`dhall diff` slightly misbehaves when diffing the following expression with
itself:
```dhall
λ(f : List Bool -> Bool) → f ([] : List Bool)
```
... producing the following diff:
```
λ(… : …
→ …)
→ …@…
- [ … ] : List …
+ [ … ] : List …
```
This is because there are two places in the `Dhall.Diff` module responsible
for comparing lists:
* Once in `diffApplicationExpression`, which compares two lists with at least
one type annotation between them
* Once in `diffPrimitiveExpression`, which compares two lists if neither one has
a type annotation
Those cases exhaustively cover all possible pairs of lists, but there was a
third (incorrect) fallback case that prematurely gave up and displayed them as
different. This fallback would trigger when applying a function to an empty
list, since the diffing algorithm wouldn't have a chance to return back to the
top-level `diffExpression` and try to compare the lists correctly in
`diffApplicationExpression`.
After this change, the diff now doesn't include a spurious difference:
```
λ(… : …
→ …)
→ …@…
…
```
The `--file` option was essentially broken since it passed in the
file path instead of the directory, meaning that transitive imports
would not be computed correctly (they'd be off by one path component).
* Find unused bindings inside nested lets
The removeUnusedBindings rule only matches the first bound variable in a
nested let block (of the form "let ... let ... in ..."). This means that
so far the linter missed cases like
let a = 0 let b = 0 in a.
This simple fix unfolds all let blocks (syntactically this means
inserting `in's everywhere) before applying the linting rules; the
LetInLet rule folds everything back together in the end. Applied to the
example from above we now return the correct result,
let a = 0 in a.
* Rewrite `lint` in a more explicit style; add comments
* Don't export implementation details
* Fix `freeIn` to correctly handle de Briujn indices
Previously we had "x@0" `freeIn` "x@1" == True (but "x@1" `freeIn` "x@0"
== False). The fix is to use subst instead of shift to change
occurrences of the given variable.
* Fix `removeUnusedBindings` to update de Bruijn indices
Whenever we remove an unused binding we need to update any references to
variables outside the let block, so that
let a = 0 let a = 0 in a@1
gets correctly rewritten into
let a = 0 in a (a = a@0)
instead of
let a = 0 let a = 0 in a@1
~> let a = 0 in a@1
~> a@1
Fixes https://github.com/dhall-lang/dhall-lang/issues/579
`Natural/fold` was misbehaving on GHCJS due to the use of `Natural`
arithmetic (for some reason). This is a problem I've seen and fixed
before, but needed to be fixed again after the migration to the new
`Dhall.Eval` normalization engine.
The fix is easy: use `Integer` instead of `Natural` for the accumulator
of the fold and the bug disappears.
Part of https://github.com/dhall-lang/dhall-lang/issues/563
This flag freezes imports in the same way as the Prelude by providing a
fallback unprotected import without an integrity check. The primary use
case for this is caching imports with a graceful fallback, which is why
the flag is named `--cache`
As Dhall's bounds don't allow base 4.13, this doesn't actually affect
anyone running in a supported configuration (i.e., without
--allow-newer). Further note that base 4.13 (i.e., GHC 8.8) isn't
tested in CI at present.
This could well be the last GHC 8.8-related change needed to
code (bounds will definitely need to be adjusted). In this case, a
Dhall release with relaxed bounds will suffice to finish off GHC 8.8
support.
However, it's also possible that dependencies might bundle together
breaking changes with 8.8 support, in which case adaptations will
still need to be made.
Fixes https://github.com/dhall-lang/dhall-haskell/issues/882
This allows users to supply the Dhall expression by path instead of via
standard input. This also ensures that transitive imports are resolved
appropriately relative to the file's path.
In other words, instead of this:
```
$ dhall <<< './foo/bar.dhall'
```
Users can now do:
```
$ dhall --file foo/bar.dhall
```
This adds a new `Dhall.Test.Util.discover` utility for auto-generating
a `TestTree` from a directory tree. This simplifies keeping up to date
with changes to the standard test suite.
- Dhall.Eval: new evaluator, conversion checker and normalizer.
There is no standalone alpha normalizer yet.
- There is a new option "new-normalize" for dhall executable, which uses
the new normalizer.
- Type checker is unchanged.
- new implementation: alphaNormalize, judgmentallyEqual, normalize
- normalizeWith takes a Maybe ReifiedNormalizer argument now, and switches to
the new evaluator whenever the input normalizer is Nothing
- QuickCheck test for isNormalized removed, because we don't support evaluation
of ill-typed terms, which the test would require.
... as standardized in https://github.com/dhall-lang/dhall-lang/pull/426
This adds two new `ToTerm`/`FromTerm` classes in order to minimize
code disruption. The main disruption is due to renaming the old
`encode`/`decode` to `encodeExpression`/`decodeExpression`
... as standardized in https://github.com/dhall-lang/dhall-lang/pull/438
This also adds `dhall-json` support for empty alternatives
In particular, this translates empty alternatives to strings encoding the alternative name
```haskell
-- ./example.dhall
let Role = < Wizard | Fighter | Rogue >
in [ Role.Wizard, Role.Fighter ]
```
```
$ dhall-to-json <<< './example.dhall'
["Wizard","Fighter"]
```
... as caught by @singpolyma
`Dhall.Binary.unApply` was failing to uncurry functions due to the presence of
`Note` constructors. This caused `dhall encode` to produce different
results from `dhall hash` since the latter was normalizing the expression
(and therefore removing `Note` constructors) whereas the former was not.
This commit fixes an issue that made the last branch of the completer
unreachable.
The second to last branch was always `True`:
```
> split (== '.') "anything"
["anything"]
```
Signed-off-by: Basile Henry <bjm.henry@gmail.com>
Fixes#832 832
This includes two changes:
* Fix a missing newline at the end of `dhall format` output when not using
the `--inplace` option
* Better align ASCII output
.. as caught by @Profpatsch in:
https://github.com/dhall-lang/dhall-haskell/pull/812#issuecomment-462134701
Before this change the location was always being reported as `(stdin):1:1`
because the `SourcedException` kept getting modified with a broader
source location in the `Note` branch of `loadWith`.
This was originally done so that alternative imports would show the entire
source span, but it mistakenly just kept bubbling up regardless of whether or
not there were alternative imports.
Instead, this includes an approximate source span for alternative imports.
The source span bounds are correct but the contents just show which imports
were alternated, even if they might have been buried in a larger expression.
For example, if the original expression had been:
```haskell
Some ./x ? None ./y
```
... then the source span for the error message would display just:
```haskell
./x ? ./y
```
... which is probably as good as it will get for now.
The format for the save file is a plain Dhall file containing REPL commands.
This will hopefully mean that the save file can be edited manually if need be
without too much effort.
Signed-off-by: Basile Henry <bjm.henry@gmail.com>
The motivation behind this change is so that users can freeze remote imports
(like the Prelude) but ignore local imports so that subsequent runs of the
interpreter reflect changes to local files and environment variables.
The reasoning behind this is that there are two primary benefits of integrity
checks:
* Improved security
* Caching
... and one downside which is that updates to those imports are not pulled in
until the integrity check is updated or removed.
However, environment variables and local file paths do not benefit from
improved security or caching, so there is only a downside to freezing them.
Specifically:
* Environment variables and local file paths are both cheap to resolve
... so they don't really benefit from caching.
To be precise, they *could* benefit from caching if the cache expression is
cheaper to parse and normalize compared to the original file. For those
cases there is still an `--all` flag to freeze all imports.
* Environment variables and local file paths are trusted
For example, when a user runs the `dhall` executable they are implicitly
trusting their filesystem which provides that executable. Similarly, when
they run `dhall` without an absolute path they are implicitly trusting that
their `PATH` environment variable has not been compromised to point to a
malicious executable.
Up until now, Dhall's threat model has always been that local imports are
trusted but remote imports are not, so this is consistent with that threat
model.
... so as far as environment variables and local files are concerned there are
only downsides to freezing them and no up-side. This is why this change
no longer freezes them.
This also renames `hashImport` to `freezeImport` for more terminology
consistency.
This is useful for integrating dhall to build systems where dhall
expressions (e.g. of type `Text`) are build products.
This also renames the `--list` flag to `--transitive-dependencies`
Fixes https://github.com/dhall-lang/dhall-lang/issues/357
This uses `Text.Megaparsec.Char.Lexer.decimal` which matches the Dhall
grammar. The previous `decimal` parser was also accepting non-standard
hex and octal literals.
This commit adds a flag for producing a machine readable listing of
dependencies of a dhall expression.
This flag can be used to integrate dhall more easily to file watchers or
build systems which require such lists of dependencies.
`dhall lint` would fail on the following expression:
```
let replicate = https://prelude.dhall-lang.org/List/replicate
in let Config = { name : Text, age : Natural }
in let Configs = List Config
in replicate 10 Text "!"
```
... because the code (incorrectly) assumed that simplifying an inner
`let` binding would preserve at least one `let` binding. However, when the
outer `let` (beginning with `let replicate`) is simplified the inner `let`
(beginning with `let Config`) simplifies down to just `replicate 10 Text "!"`
which has no `let` binding at all, leading to a pattern match failure.
This change fixes that by extending the code to correctly handle that case
with an exhaustive pattern match.
Related to: https://github.com/dhall-lang/dhall-lang/issues/343
According to the standard a non-empty list with a type annotation should
not be encoded with the type annotation alongside the list elements. The
type annotation should be parsed and encoded as a separate constructor.
A refactor of the Nix build accidentally removed the `-Werror` flag, which
caused some warnings to get past CI. This change fixes that and removes the
warnings.
The Haskell implementation was not matching the specification for
type-checking union types:
* The inferred type of union types was (incorrectly) always returning `Type`
* Unions of mixed alternative types were not being properly rejected
I also discovered several mistakes in the error messages, which I fixed along
the way.
... as standardized in https://github.com/dhall-lang/dhall-lang/pull/307
Note that this a breaking change because:
* Newlines are now mandatory after the opening `''` of a multi-line
literal
* The spaces before the trailing `''` count towards the leading
indentation computation
* Test with QuickCheck that an expression is always the same as itself
I noticed cases where the diffing code returns that an expression is not the
same as itself (for example, an empty list). This commit adds a QuickCheck test
to illustrate it, and maybe other cases.
Sadly, I had to expose some internals from Dhall.Diff for the test, which makes
the interface less nice.
* Fix diff so that an empty list is the same as itself
This adds three new Nix build products:
* `try-dhall-static` - The static assets for "Try Dhall"
* `try-dhall-server` - A script which serves the static assets for local
debugging
* `tarball-try-dhall` - A tarball of the static assets
This is powered by a new small `dhall-try` package which is also included
in this change.
* Autocomplete repl commands
* Autocomplete imports (file and environment)
* Improve autocomplete separators
* Autocomplete records fields
* Remove use of TypeApplications
* Improve file path completion
* Autocomplete union alternatives
* Import missing (<>)
* Improve parsing of assignment in repl commands
* Remove no-longer-used `foldl'`
* Fix record field autocomplete and improve command autocomplete
* Fix definition order in repl environment
* Autocomplete variables in scope and all reserved identifiers
The motivation for this change is so that more code type-checks
within a REPL session.
For example, before this change the following session fails:
```haskell
⊢ :let Either = λ(a : Type) → λ(b : Type) → < Left : a | Right : b >
⊢ :let Example = < Left : Natural | Right : Bool >
Example : Type
⊢ Example.Left
Example : Type
Error: Not a record or a union
Example.Left
(stdin):1:1
```
After this change, the same session works:
```
[⊢ :let Example = < Left : Natural | Right : Bool >
Example : Type
⊢ Example.Left
λ(Left : Natural) → < Left = Left | Right : Bool >
```
The difference is in how we incorporate prior definitions when
type-checking code. Before this change prior definitions were added
to the context, which leads to type-checking failures because type
synonyms don't work when used in this way. After this change we
incorporate them as extra `let` definitions (same as normalization).
This is essentially a revert of 25d86e8e5d
The primary motivation for this is so that paths beginning with `../`
don't get reformatted to begin `./../`.
The secondary motivation for this is to more closely match the standard.
There are fewer special cases in the logic if there is a dedicated
constructor for the `Parent` case.
Fixes#714
This adds a `--json` flag that `dhall decode` and `dhall encode` can use to
read/write the equivalent JSON representation of the CBOR. This comes in
handy for the parsing compliance tests which use this CBOR-as-JSON as the
standard representation of the abstract syntax tree.
Now that `constructors x` is the same as `x` we can have `dhall lint`
safely strip all uses of `constructors` in preparation for removing the
keyword entirely.
Fixes#509
The `Dhall.Import.HTTP` module had logic for pretty-printing HTTP error
message, but this logic wasn't being used anywhere! This change fixes
that and also polishes the error messages a little bit.
* Add dotgen as a dependency
Signed-off-by: Basile Henry <bjm.henry@gmail.com>
* Build up dot graph while resolving imports
Signed-off-by: Basile Henry <bjm.henry@gmail.com>
* Add --dot option to resolve in CLI
Signed-off-by: Basile Henry <bjm.henry@gmail.com>
* Handle diamond dependencies in dot graph
* Refactor dot graph generation
This fixes an apparently very old bug in import caching caught by @basile-henry
Before this change the import resolution algorithm was:
1. Retrieving the cache
2. Transitively resolving all imports
3. Setting the new cache to be current import insert into the cache retrieved in
step 1
The bug is that all of the transitive imports resolved in step 2 added
entries of their own to the cache and those new cache entries were being
clobbered by step 3.
The fix is simple: don't use the cache retrieved in step 1 to compute
the updated cache in step 3. Rather, use `modify` instead of `put` to
create the new cache so that we preserve newly-added cache entries.
`dhall lint` was incorrectly deleting `let` bindings that are being used
due to not checking other `let` bindings within the same multi-`let`
expression for free variable occurrences.
This change fixes that and adds the first regression test for `dhall
lint`
Fixes#692
The standard permits a user to access a constructor from a type stored inside
a record, but the Haskell implementation had a mistake which prevented this.
Specifically, the Haskell implementation was not normalizing the union type
as the standard specified before attempting to access the constructor, leading
to an unexpected type error.
Fixes https://github.com/dhall-lang/dhall-lang/issues/267
According to the standard, Unicode characters up to `0x10FFFF` do not
require escaping. See:
33cab24f8e/standard/dhall.abnf (L192)
... so we can preserve them when pretty-printing Dhall expressions.
Note that the current code still does not comply with the standard for Unicode
characters beyond `0x10FFFF`, but I'll defer fixing that to a subsequent
change.
The issue was that the parser was attempting to parse // first, which
will succeed on the prefix of //\\, then the parser will get an error
because it expects a sub expression but the input is \\.
The motivation for this change is:
* To catch build failures in downstream packages whenever we make a breaking
change to the `dhall` API
* To reduce the amount of work I need in order to cut a release for all of
these packages
* To better share Nix/CI-related logic between the projects
Note that I have not yet migrated `dhall-nix` in. I'm waiting for
https://github.com/dhall-lang/dhall-nix/issues/17 to be fixed since
`dhall-nix` is incompatible with later versions of `megaparsec` due to
`hnix`.
