This one is obvious, everything exists within a game.
The library project is our equivalent of nuget packages.
You can create a common library of code and publish it as a package. This enables you to re-use the same code over multiple games by adding the package as a reference.
Once you have made your library public you should be aware that if you remove or change the public API of the assembly you risk breaking projects that are referencing it.
Addons allow you to build on top of and modify existing games.
Addons can target a specific game, or work universally across all games. Though not every game will support loading addons.