Initialization of a user defined database, username, and password using environment variables #329
Comments
johnwyles
changed the title
|
Agreed! There should be an easy way to spin up a container, with a new database, username and password - all specified in |
|
I understand your confusion. We only create a user with the See also,
As for the database not existing, that is just how MongoDB works as I have explained before. If nothing is inserted the database does not exist (or conversely every database exists, you just have to use it). I'm mostly against adding more environment variables for creating a non-admin user, especially if it breaks backwards compatibility. We try to keep the images as close to upstream releases as possible with minimal maintenance and the entrypoint script is already very complex. A simple Dockerfile: FROM mongo:4.0
COPY custom-user.sh /docker-entrypoint-initdb.d/and a short #!/bin/bash
set -e;
# a default non-root role
MONGO_NON_ROOT_ROLE="${MONGO_NON_ROOT_ROLE:-readWrite}"
if [ -n "${MONGO_NON_ROOT_USERNAME:-}" ] && [ -n "${MONGO_NON_ROOT_PASSWORD:-}" ]; then
"${mongo[@]}" "$MONGO_INITDB_DATABASE" <<-EOJS
db.createUser({
user: $(_js_escape "$MONGO_NON_ROOT_USERNAME"),
pwd: $(_js_escape "$MONGO_NON_ROOT_PASSWORD"),
roles: [ { role: $(_js_escape "$MONGO_NON_ROOT_ROLE"), db: $(_js_escape "$MONGO_INITDB_DATABASE") } ]
})
EOJS
else
# print warning or kill temporary mongo and exit non-zero
fiCombine that with automated builds (https://docs.docker.com/docker-hub/builds/) and repository links (https://docs.docker.com/docker-hub/builds/#repository-links) and it's reasonably easy to have an up-to-date image built $ docker run -d -e MONGO_INITDB_ROOT_USERNAME=root -e MONGO_INITDB_ROOT_PASSWORD=supersecret -e MONGO_INITDB_DATABASE=notadmin -e MONGO_NON_ROOT_USERNAME=normal -e MONGO_NON_ROOT_PASSWORD=secret --name some-mongo custom-mongo
$ docker run -it --rm --link some-mongo mongo:4.0 mongo -u normal -p secret some-mongo/notadmin
MongoDB shell version v4.0.6
connecting to: mongodb://some-mongo:27017/notadmin?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("0941eaa4-9180-4012-a24c-60fe8369b06a") }
MongoDB server version: 4.0.6
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
http://docs.mongodb.org/
Questions? Try the support group
http://groups.google.com/group/mongodb-user
> use admin;
switched to db admin
> db.users.findOne();
2019-02-05T23:57:10.838+0000 E QUERY [js] Error: error: {
"ok" : 0,
"errmsg" : "not authorized on admin to execute command { find: \"users\", filter: {}, limit: 1.0, singleBatch: true, lsid: { id: UUID(\"c86d23b2-9ed4-4799-9d3e-74f327366550\") }, $db: \"admin\" }",
"code" : 13,
"codeName" : "Unauthorized"
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
DBCommandCursor@src/mongo/shell/query.js:708:1
DBQuery.prototype._exec@src/mongo/shell/query.js:113:28
DBQuery.prototype.hasNext@src/mongo/shell/query.js:288:5
DBCollection.prototype.findOne@src/mongo/shell/collection.js:260:10
@(shell):1:1
> use notadmin;
switched to db notadmin
> db.users.findOne();
null
This seems clear to me (emphasis added):
@tianon has a PR for improving the user section: docker-library/docs#1418 |
|
@yosifkit Thank you for that response. Given the majority of developers are writing their code and initialization and configuration such that they assume (whether rightly or wrongly) that their database is there and a username and password they know is able to talk to and operate on it I think it would be a great addition to the image. Given your excellently laid out example above I think this should be baked in and not putting onus on the users of this image to essentially all have copies of the script you mentioned running around in it's various permutations. What do you say to including it in the image:
I think this is excellent if you were driving at building this into the image and see no reason not to since 99% of cases users will want a pre-initialized database with a username and password - putting this in documentation will help but having the extra step of them essentially copy/pasting your script so they can get what would be expected behavior of an image with an upstream project like this where the vast majority of developers and users are not having to write the database / username/password initialization into their code after connecting with a And of course simply adding your |
|
I understand the reluctance to add functionality to working software. I also understand @yosifkit's response, and I'm using his advice in my own code. But this functionality is obvious and necessary - as evidenced by all those who complained about it in #174 and StackOverflow. We assumed it to work the way we've described, because that is the obvious need. One of the reasons we use docker is the promise of "just spinning up" a new container that is ready for action. Extra steps and config (that exists in multiple places, and liable to fall out of sync) are an extra burden, which are easily avoided by the (backwards compatible) proposed additions. |
|
I have submitted PR #331 as outlined by @yosifkit comments and code example as well taking into account his comments:
This does not break backwards compatibility and is a fully optional feature added for expected user behavior as itterated above by @lonix1 @johnwyles in this issue and numerous others in issue #174 (comment).
While more mature software projects will certainly require And then I will just echo all the points in: #329 (comment)
I will also put together a PR for the documentation update if we can proceed on this 👍 |
|
I have also added the documentation for this in PR docker-library/docs#1422 in the documentation as well. |
|
BUMP What did we think about PR #331 and docker-library/docs#1422 to provide users with expected behavior, backwards compatibility, and optional parameters but that when present provide the expectation the user requires for database initialization? Let me know if the PR needs a bit of modification but the discussions here seem to have all been addressed in the PR and documentation PR that provide comfortably merging this in. |
|
Actually, I've posted a comment in a different issue, but it seems to be very mongo initialization process related, so that will leave it here also #323 To summarize: there are problems with |
|
@npiskunov, on your issue it looks like we just need to account for the default On a related note, this also means that the PR for this issue(#331) has another edge case. The non-root user values cannot work when running as a config server since it cannot create a user outside of
Finding more edge cases like this is why I am so hesitant to add more environment variables. Admittedly this means that, apart from #331, we might want to note that limitation for |
|
@yosifkit Did I get right, that changes in #341 fix behavior of MONGO_NON_ROOT_USERNAME in configsvr by pointing user create on correct DB (admin)? In this case Regarding #341. In my opinion changes described (mongo initialization variables) become clearer, but still not fully clear :) It would be perfect if there was e.g. compose.yml illustrating minimalistic sharded cluster composition with security (3-members RS for shard, 3-members CSRS, single mongos. Root user / Security Key file / sample DB initial seed); |
|
Why this example is not in the documentation?
|
|
to echo @carloschneider and @lonix1 - @yosifkit I think there is an "expectation" that the legwork not be on the implementer or user of this to have some expectation to add a lot of custom work and have to wade through these github PR's and Issues to find out how to implement what should otherwise be straightforward behavior one would expect - I think pushing PR https://github.com/docker-library/docs/pull/1421/files (docs) (which was closed) but also something similar or the PR I originally filed on this: https://github.com/docker-library/mongo/pull/331/files will resolve a lot of headache you clearly can see causing people to pull their hair out only to find they need to write some custom code and implementation with what should otherwise be expected or "obvious" behavior - sorry for getting back to this so late as I have been caught up in a lot of other work which let this linger for longer than it should have |
|
This is exactly what my development environment needs! I want to test my application's connection to a "production like" mongo instance with authentication and so on. I'm able to do this with my local docker-compose utilizing the custom made This feature could provide a centralized way to look up your mongo credentials and your application credentials in a clear and consistent manner in your docker-compose.yml or other configuration files. Cheers for the efforts! |
|
Can someone post an example of how to run the docker commands without --link and using --network? I can't seem to get my nodeJS app to connect to the mongo container. |
|
Why is a custom non root user still not a part of the default Dockerfile? People need it and people end up implementing it themselves with It could save so many hours of work if the maintainer would do it once and folks can just add some environment variables instead of creating an additional dockerfile + script... |
|
You can use the standard $ mkdir db && sudo chown 777 db
$ docker run --rm -dit -v $PWD/db:/data/db -v /etc/passwd:/etc/passwd:ro --user 1000:1001 --name mongo mongo
6c4a4e74b314a2a01893bc0b7405106db3100db37e9ad466772f56f2f88ec2b2
$ echo $UID
1000
$ docker exec -it mongo bash
groups: cannot find name for group ID 1001
rei@6c4a4e74b314:/$ echo $UID
1000 |
|
Bumping this because I find it a little frustrating that I'm blocked because I can't initialize a mongo container with a usable database + username/password pair. user @lonix1 stated it better than I could:
My use case is simple: Can I connect to my mongodb container with a username and password on application start-up and begin using my target database without issue from a single docker-compose.yml file? No, because environment variables don't cover this extremely basic configuration. Has this been reconsidered since 2019, @yosifkit? UPDATE: Went with |
|
Can we please get some documentation added and add the init script to run as part of the entrypoint? |
|
@CodePint That is the point of the ticket so not sure if you were just echoing it, but you can see for help: Other than getting the maintainers to merge those tickets you are going to be stuck where a lot of folks are on these two open PRs. |
|
I added the following to my # ! update mongodb user for authentication
sleep 10
export $(sed 's/[[:blank:]]//g; /^#/d' .env | xargs)
docker exec -it mongodb bash -c "mongo $MONGO_DB \
-u $MONGO_USERNAME \
-p $MONGO_PASSWORD \
--authenticationDatabase admin \
--eval 'db.createUser({user: \"$MONGO_USERNAME\", pwd: \"$MONGO_PASSWORD\", roles:[{role:\"userAdminAnyDatabase\", db: \"admin\"}]});
db.grantRolesToUser(\"$MONGO_USERNAME\",[{ role: \"root\", db: \"admin\" }]);'"I'd like to leave it here in case some poor soul is in need of it it was my first time using mongodb the other day and I went for docker straight away(big mistake). I spent the better part of 2 days trying to figure out a way around all this that didn't involve volume mounting or overriding the entry_point.sh file it's not the cleanest solution but it does the job for now |
|
Thanks everyone for your effort and documentation. I agree that it would be helpful to have an easy way to use docker to set up a local connection with default values without adding bash script files. |
|
After spending all evening last night and this morning trying to figure out why these environment variables "were not working", I finally found this discussion. I would like to add my support for this feature. I'm considering moving away from this image and using Bitnami's image instead for this very reason. |
@johnwyles
@draeder I will be using their image in the future. As an aid to frustrated developers who will visit this page over the coming years, I've included the relevant info for the Bitnami MongoDB image below: GitHub Repository: Running the docker image with "a user defined database, username, and password using environment variables": # Creating a user and database on first run
# You can create a user with restricted access to a database while starting the container for the first time.
# To do this, provide the MONGODB_USERNAME, MONGODB_PASSWORD and MONGODB_DATABASE environment variables.
$ docker run --name mongodb \
-e MONGODB_USERNAME=my_user \
-e MONGODB_PASSWORD=password123 \
-e MONGODB_DATABASE=my_database bitnami/mongodb:latest# or by modifying the docker-compose.yml file present in this repository:
services:
mongodb:
...
environment:
- MONGODB_USERNAME=my_user
- MONGODB_PASSWORD=password123
- MONGODB_DATABASE=my_database
... |
|
For everyone running into issues using the script by @carloschneider: The script is not copy/paste, as a noop is needed in the else (at least on my machine): |
|
As someone who's spent the last day or more, trying to make this container work in Azure Container Instances / App Services / Container Apps, I'm laughing at how ignorant I am and how bad the documentation is. I've inserted the required MONGO_INITDB_ROOT_USERNAME / MONGO_INITDB_ROOT_PASSWORD and MONGO_INITDB_DATABASE, got the container working with an Azure File Share and then spent hours debugging why the user and database hasn't been creating. All of the files are present in the Azure File Share though, indicating that persistent storage will work. Thank you to everyone for providing the missing steps. It seems there's no real way to have this container build out of terraform with the required .js files to actually create the user/db. It's a shame the scripts aren't included by default. |
|
It's been more than 3 years and it's absolutely funny how such a simple thing is so misleading for many developers wanting to quickly spin up a db instance. The frustration is more about a simple set of environment variables not working as expected, and developers wasting a ton of time in diagnosing why the connection isn't going through. |
correct, just killed 3 hours until figure it out, then went here to write a comment to support this feature (bug), so that in next 3 year someone fix it, and my children would not be as angry as I am right now. |
|
Please add this, it would be so much easier and logical to have a way to create the image with a default DB, User, Password. Thanks for mentionning bitnami image in the answers above!! |
|
How much time and money could be saved with this issue and script are included in the docs or variables name be cleaner and logical. |
This issue (I would call it a bug but perhaps it is a feature request) is that users would like to a la
docker-compose.ymland/or environment variables be able to set a database with a username and password they specify upon launch of the image.Background:
This issue was filed #174 and closed because the behavior of a PR #145 was mentioned as the solution. What #145 actually does and what users expect are entirely different. What PR #145 does is set a user with elevated permissions (i.e. "root" user) that has superuser access to the entire MongoDB instance (as mentioned in #174 (comment). However what most users expect from these environment variables is that a database they specify is initialized with the username and password they have set. It is confusing that these environment variables (
MONGO_INITDB_DATABASE,MONGO_INITDB_ROOT_PASSWORDandMONGO_INITDB_ROOT_USERNAME) pertain to only setting a user with the rolerooton the databaseadminand initializing an user specified database for.jsand.shscripts in/docker-entrypoint-initdb.d/to be run against.Proposal:
We should make the environment variables very explicitly named in what they do in addition to adding others for the behavior I think most users come to expect when reading the variable names. Since it is the case most users would like their instance initialized with a database of their specification we should add this feature to meet that expectation.
MONGO_INITDB_ROOT_USERNAMEandMONGO_INITDB_ROOT_PASSWORDMONGO_INITDB_DATABASEas it is misleadingMONGO_INITDB_ROOT_DATABASEand allow it to override the hardcodedadmindatabaseMONGO_USERDB_ADMIN_USERNAME,MONGO_USERDB_ADMIN_PASSWORD, andMONGO_USERDB_ADMIN_DATABASEMONGO_INITDB_ROOT_USERNAME,MONGO_INITDB_ROOT_PASSWORD, andMONGO_INITDB_ROOT_DATABASEwill be used for therootrole to MongoMONGO_USERDB_ADMIN_USERNAME,MONGO_USERDB_ADMIN_PASSWORD, andMONGO_USERDB_ADMIN_DATABASEwill be used to initialize a user specified database.jsand.shscripts a user supplies in/docker-entrypoint-initdb.d/will be executed againstMONGO_USERDB_ADMIN_DATABASEReasons for change:
MONGO_INITDB_ROOT_PASSWORDandMONGO_INITDB_ROOT_USERNAMEare only used for theadmindatabaseMONGO_INITDB_DATABASEdoes is have operations used against it whenever a user has dropped in.jsor.shscripts into/docker-entrypoint-initdb.d/. This unclear unless you look atdocker-entrypoint.shin this repository and no where clearly stated in the documentation as such/docker-entrypoint-initdb.d/which then places burden on the user to maintain therootrole credentials in environment variables which live separately from a custom.jsor.shscript which they have to volume into the imageReferences:
Involved Persons:
@mmi-rperez
@tianon
@vutran1710
@yosifkit
@lonix1
@johnwyles
The text was updated successfully, but these errors were encountered: