---

---

Environments

What is a environment

• A list of “installed” packages¹ at certain versions
• sometimes includes the operation system

Reproducible vs. Replicable

• Reproducible: Someone else can get the same results given your code + data
• Replicable: Someone else can repeat the whole study on new data

Why do I need it?

• Version control
  • managing multiple projects
  • testing out things
  • your Toolbox needs other packages than e.g. your test-environemnt
• Reproducibility
  • other researchers want to know what packages to install
• Collaboration
  • you want to work on the same data / codebase

Environments in Julia

Every folder with an Project.toml file has it’s own environment (see below)

The “base” environment is active by default:

(@v1.9) pkg>

Keep this as empty+tidy as possible!

Tip: (you could also start julia by julia --project=".")

Typical commands

activate

Use activate . or activate ./path/to creates a new Project.toml in the selected folder (. means current folder), or activates it, if it already exists.

status (st)

Shows the currently installed packages

add

Multiple ways to add packages to the Project.toml:

• add UnicodePlots
• add https://github.com/JuliaPlots/UnicodePlots.jl
• specify branch: add UnicodePlots#unicodeplots-docs
• specify version add UnicodePlots@3.3
• add ./path/to/localPackage

Note

Folders have to be git-repositories, see below. Probably better use develop

remove (rm)

remove package from Project.toml (not from ~/.julia, use gc - garbage collect for this)

develop

• dev --local UnicodePlots
• dev ./Path/To/LocalPackage/

Note

You can’t select a branch with dev and need to do it manually

Note

You are asking for the difference of dev ./Path/Package and add ./Path/Package? Good question! dev will always track the actual content of the folder - whereas add will make a “snapshot” of the last commit in that folder (has to be an git for add!). And you have to use ]up to actually update to new changes

pin / free

You can pin versions of packages, so that they are not updated. Unpin with free - also undo develop by using free

instantiate / resolve

instantiate setup all dependencies in the given Project.toml+Manifest.toml

resolve update the Manifest.toml to respect the local setup

Project vs. Package

	Project	Package
installable/reuse?	✅	❎
should be reproducible	✅	❎
produces something?	✅	❎
compatabilities declared?	❎	✅
formal requirements in julia?	❎	✅

Project.toml & Manifest.toml

📄Project.toml

The “big picture”: keeps track of user-added dependencies (+ compatabilities + header)

[deps]
PythonCall = "6099a3de-0909-46bc-b1f4-468b9a2dfc0d"
RCall = "6f49c342-dc21-5d91-9882-a32aef131414"

📄Manifest.toml

The “details”: keeps track of all versions of all dependencies, and dependencies of dependencies

julia_version = "1.9.2"

[[deps.AbstractPlutoDingetjes]]
deps = ["Pkg"]
git-tree-sha1 = "8eaf9f1b4921132a4cff3f36a1d9ba923b14a481"
uuid = "6e696c72-6542-2067-7265-42206c756150"
version = "1.1.4"

[[deps.ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"
version = "1.1.1"

[[deps.BibInternal]]
git-tree-sha1 = "3a760b38ba8da19e64d29244f06104823ff26f25"
uuid = "2027ae74-3657-4b95-ae00-e2f7d55c3e64"
version = "0.3.4"

[...]

Projects in Julia

Formally, projects don’t have specific requirements. You should activate an environment (Project.toml+Manifest.toml) in the main folder though. I recommend the following minimal structure:

• ./src/ - all functions should go there
• ./scripts/ - all actual scripts should go here,
• ./README.md - Write what this is about, who you are etc.
• ./Project.toml - Your explicit dependencies
• ./Manifest.toml - Your implicit dependencies + versions <– this makes it reproducible!

Tip

One recommendation is to use DrWatson.initialize_project([path]) to start a new project - it will generate a nice folder structure + provide some other helpful DrWatson.jl features.

Click to expand the full datastructure

│projectdir          <- Project's main folder. It is initialized as a Git
│                       repository with a reasonable .gitignore file.
│
├── _research        <- WIP scripts, code, notes, comments,
│   |                   to-dos and anything in an alpha state.
│   └── tmp          <- Temporary data folder.
│
├── data             <- **Immutable and add-only!**
│   ├── sims         <- Data resulting directly from simulations.
│   ├── exp_pro      <- Data from processing experiments.
│   └── exp_raw      <- Raw experimental data.
│
├── plots            <- Self-explanatory.
├── notebooks        <- Jupyter, Weave or any other mixed media notebooks.
│
├── papers           <- Scientific papers resulting from the project.
│
├── scripts          <- Various scripts, e.g. simulations, plotting, analysis,
│   │                   The scripts use the `src` folder for their base code.
│   └── intro.jl     <- Simple file that uses DrWatson and uses its greeting.
│
├── src              <- Source code for use in this project. Contains functions,
│                       structures and modules that are used throughout
│                       the project and in multiple scripts.
│
├── README.md        <- Optional top-level README for anyone using this project.
├── .gitignore       <- by default ignores _research, data, plots, videos,
│                       notebooks and latex-compilation related files.
│
├── Manifest.toml    <- Contains full list of exact package versions used currently.
└── Project.toml     <- Main project file, allows activation and installation.
                        Includes DrWatson by default.

Packages in Julia

Several thousand packages exist in Julia already. Take a thorough look before starting something new!

Minimal requirements for ]add to work

Minimal structure

One git-repository containing:

• ./src/MyStatsPackage.jl
  • (module MyStatsPackage)
• ./Project.toml
  • name = "MyStatsPackage"
  • uuid ="b4cd1eb8-1e24-11e8-3319-93036a3eb9f3"
  • ([compat] entries)
  • (version= "0.1.0")

Additional requirements to register

Julia supports many registries (you can host your own!), which are just fancy GITs that index what version is available at what git-url for each registered package.

The default registry is JuliaRegistries/General.

To register at the general registiry, you need additionally::

• [compat] entries for all dependencies
• a version=
• a supported license
• Some restrictions on the name (e.g. nothing with Julia, only ASCII, etc.)

Let’s generate our first package!

] generate MyStatsPackage

Adding dependencies

]activate ./path/to/MyStatsPackage
]add ProgressMeter
]compat

1

let’s directly add a compat entry for ProgressMeter

Semantic Versioning

Following semver - three parts:

v2.7.5

means:

• Major 2

• Minor 7

• Bugfix 5

• Bump Major if you propose backward-breaking changes

• Bump Minor if you only introduce new features -Bump Bugfix if you, well, fix bugs

Special case:

v0.37.1

Means package is in development and not stable. Bump Major if you release it Bump Minor for breaking changes Bump Bugfix if you fix bugs or release new features

Compat entries

compat entries define with what versions your package is compatible with

[compat]
AllMinorReleases1 = "1"
AllMinorReleases2 = "1.5"
AllMinorReleases3 = "1.5.3"
ExactPackage = "=1.5.6"
MultiVersionexample = "0.5,1.2,2"
DevelopPackage = "0.2.3"

1

[1.0.0-2)

2

[1.5.0-2)

3

[1.5.3-2)

4

[1.5.6]

5

[0.2.3 - 0.3)

As you can see, develop version (version < 1) are treated a bit special in Julia, and different to semver. Read more here

Warning

keep the compat list in alphabetical order - github-actions might behave very strange else.

Internals of a package

The file ./src/MyStatsPackage.jl should contain:

module MyStatsPackage
using ProgressMeter
include("src/stats_functions.jl")

export sum
export mean, tstat

end

1

We use GLMakie as a simple example as you need it on Wednesday again anyway - it does take a while to install though!

Now we are ready to use the package from a different environment

]dev ./path/to/MyStatsPackage

Footnotes

1. libraries, dlls, .so etc.