Unfortunately mongorestore doesn’t let you run queries on the to-be-restored documents, indeed. The best it can do is include/exclude specified databases and collections.
mongodump, however, does let you run queries - maybe you could use that to narrow down which documents are stored in the dump? 
Not an option, no. Our tech team backups entire product databases with all data (nothing else would be reasonable) but we as support only need parts of the data to respond to support cases.
My import is running since 3 hours, did not complete successfully. And thats my… 10th try, trying different combinations of import options (no validations, no indexes etc). So dependability of that import feature is… problematic. I’m currently not able to respond to that support case properly.
Any ideas what I can do to finally succeed?
Unfortunately in the case of mongodump and mongorestore, there’s not much that Studio 3T itself does, it’s really mostly done by those two tools.
You could try running the command manually on the command line (the command can be copied from the status log, you’ll have to add the needed passwords though) and see if that’s any better. If not, this may just be a mongorestore issue. If it is better, there may be a bug in Studio 3T.
You could also try configuring a different/newer version in Preferences - MongoDB Tools. It could be that this is fixed in a newer release of mongorestore which we aren’t bundling yet (we currently bundle 100.5.4, the newest release seems to be 100.7.0).
It’s defenitely not a Studio3T problem, I know that by now.
I’ve tried Studio3T (mongo-tools 100.5) restore in both MongoDB v5 and v6 server as well as mongo-tools 100.7 (mongorestore.exe via CMD with parameters) on v5 and v6 DB.
Command for example: mongorestore --verbose --nsInclude=schulcloud.files --drop --noIndexRestore --stopOnError --bypassDocumentValidation --archive="c:\Users\Administrator\Desktop\Backups u Doku\JHD-45913\mongodb-schulcloud-20230413.gz" --gzip --numInsertionWorkersPerCollection=1
the restore starts but halts somewhere in the process, always at a different progress:
{"t":{"$date":"2023-05-12T15:02:08.560+02:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"Checkpointer","msg":"WiredTiger message","attr":{"message":"[1683896528:559770][7284:140716900309808], WT_SESSION.checkpoint: [WT_VERB_CHECKPOINT_PROGRESS] saving checkpoint snapshot min: 72935, snapshot max: 72935 snapshot count: 0, oldest timestamp: (0, 0) , meta checkpoint timestamp: (0, 0) base write gen: 689261"}}
Different story, back to thread topic:
I really like the new Collection History feature because it takes a step into the right (in terms of what I desire) direction.
Suggestions:
)Keep up the good work!
Hi Hannes!
I’ve passed this feedback to our team who built this feature. We really appreciate these valuable comments. We’ll look into how to improve the experience of finding the Collection History for a start. Deletion via IntelliShell or Aggregation isn’t something we were considering currently, but we will certainly discuss it nonetheless. Adding Edits to this feature is in the pipeline already so we look forward to hearing your review of this once it becomes available in future
Maybe I’m missing a feature and just doin’ it wrong. Otherwise:
Suggestion: It would be good to be able to use file-based input data in IntelliShell scripts. We often get big csv data files and have to check/modify/create/delete data in our databases based on the given data. If Studio could “load” a csv file and make it accessible in IntelliShell via a variable, that would be awesome.
What we currently do: Either by hand edit CSV as text to form a valid multi-level array to copy directly into the IntelliShell script which uses this array. Or put csv lines as text into IntelliShell and split it there. Both ways do have stumbling blocks and problems, depending on the values/texts inside.
I’m afraid I can’t share CSV content but maybe this helps to understand our current way how to handle it (one of the two above ways):
I think one of the numerous CSV-parsers as a built-in Studio feature would do a better job and maybe the fields/values would get accessible in a nicer way than “var[x]” without having to code it myself in every script.
Good suggestion, that might be worth an investigation!
Here’s something to tide you over though: mongosh is built on top of Node.js, so you can also require external modules (although they need to be installed first), which should then allow you to parse the CSV file “the Node.js way”: node.js - Parsing a CSV file using NodeJS - Stack Overflow
Suggestions:
The first suggestion already has a ticket, but I can’t promise a release date (yet).
The second point has been under consideration for a long time, with an even more generic approach in that we would just let the user supply a command to establish the SSH tunnel. Unfortunately this won’t be ready for a while, I’m afraid, but it’s definitely still on our radar.
It’s been a while! 
You know, in this thread I’m bitching a lot, but don’t get me wrong here: Studio3T is awesome, works very well, does have tons of features and is an invaluable working tool. I just want you to imporve even further!
With that in mind, here are some things that built-up in the last months, I just coulnd’t find enough free time to write it down. Until now.
IntelliShell problem: Print adds two(!) new lines (1. go to new line 2. add blank line) after the specified output. Multiple prints without blank line in between only possible with one print + multiple \n chained or workaround: Shell Output avoid new lines - #5 by SchurigH
But if you look at my 6th point regarding switch case - there all prints come without newline at all… wtf?! Just noticed that.
IntelliShell suggestions: There is a cool feature here: “Run selection (F9)” - it’s nice to test parts of a bigger script this way instead copy pasting everytime into different IntelliShell instances. BUT: If I’m using variables (which I do, of course), I always have to copy paste all variables right before the tested part to also select these. Otherwise the code won’t work. It would be nice to either be able to set global IntelliShell variables or keep variable values between runs.
Visual Query Builder problem: One of my first reported problems still exist: The yellow popup on ObjectID-Drag&Drop actions: My Suggestions / Bugs Thread - it still annoys me.
IntelliShell error detection seems broken:
console.log("test")
/db.getCollection("users").updateMany({ schoolId: fromSchool }, { $unset: { roles: "" } })
db.getCollections("users").updateMany({ schoolId: toSchool }, { $unset: { roles: "" } })
contains three errors: Slash in front of line 2, “getCollections” in line 3 and unknown variables in these lines. No errors detected by IntelliShell. Same for a coworker with this code. Console.log can also be print(…), same result.
If I remove line 1, the slash as 1 error is detected, no other errors. If I then remove the slash, no errors are detected but run the code and
Uncaught
ReferenceError: fromSchool is not defined
Uncaught
TypeError: db.getCollections is not a function
will be the output. If you take my example code from above and put a “;” at the end of console.log, everything will be an error.
I have bigger scripts where no errors are detected anymore even if there are dozens.
It is not possible to save a new IntelliShell script as an existing one (and overwrite that). To work around you can manually enter the name of an existing script into “Script name”, it will recognize it correctly. After that has been saved once, you can select existing scripts and overwrite them as you would expect it to.
Switch case break won’t work in IntelliShell scripts? Switch will not enter all cases but once its in a case, it will not exit the switch on break as you would expect:
More stuff will come tomorrow.
Alright, let’s continue:
Talking about this Document JSON Editor… Go to a document in JSON View, select some content from it (no matter what) and press Strg+J → JSON Editor opens and your selection is kept / selected again 
… BUT: Now select smth in JSON View, rightclick on it → Document → Edit Document (as JSON)… → same dialog, no selection pre-made
Talking about JSON view… Most of my time I’m using table view, lot’s of data analysed with one blink of the eye. But for some deep nested stuff I go to JSON view, again lot’s of data with one blink of the eye, but more complicated. If I then open a new collection tab, this JSON view is kept. I know, it’s somehow consistent to keep the last view but in this case it’s no good. I almost can’t think of any useful reason to look at an entire collection as JSON view, that seems not useful. Maybe add a “default view for new collection tabs” setting where I can set table view as default and it always opens that way? I’m fine with a “last used view” option within that setting that keeps the current behaviour.
Talking about UI functionality: Export → CSV → will scan your export data for fields and display them to select/unselect and… … to sort it. But no, this is not possible to be honest:
… I’m laughing so hard just how bad this is. There is no drag&drop, there is no “move to the top/bottom” and moving a field just 1 time takes like 8 seconds. No, this is not usable and it’s sad because we work massively with CSV exports for our customers. Please, Studio3T can really do better than that 
Talking about functionality ( 
): I wish for a “re-open last closed tab” feature, set to Strg+Shift+T (just like in all browsers this reopens the last tab), open Tasks (currently this shortcut) can go away! 
It would be nice to re-open the last closed tab with all it’s settings/queries, maybe even the page that you were on. Maybe it would also re-open IntelliShell tabs even with unsaved script code in it - just like Studio3T boot-up does it when it opens all previously opened tabs.
I mentioned that in another topic I think: It would be nice if Import data → BSON - mongodump archive → was able to import multiple collection with the “import data only for collection” option. Currently (last time I checked) there can only be one collection and if you want to import 10 out of 40 collections of a database backup, this gets tedious. I know that this is some kind of mongorestore command restriction but Studio as UI could also chain multiple mongorestore commands for every collection after another to allow such feature.
So… that feels good enough for now.
I wish you all a very successful new year, stay healthy and motivated, all the best!
Hannes
Dearest Hannes! Thank you so much for this epic list, and “between the years” to boot! I’ve passed this onto the team who look into user feedback and we will definitely update you on this. Happy new year, and we look forward as always to hearing more from you 
Friday again
In Preferences → Appearence → Colors → if you want to edit a Custom Color you always have to start from scratch choosing this color. If you want your old color just slighty changed, you will have to use a color picker tool to get your old color info. Overcomplicated.
In Aggregation Builder the Query history does only show executions that you made with the “Run” (Run full pipeline (F5) button or within the “Pipeline Output” tab which also executes the entire aggregate. If you test your aggregation only step by step (with the ingenious “stage input/output” feature), this won’t show. I lost a new aggregation because I built it in this Builder, executed only parts of it while building, Studio3T crashed and when starting again, no sessions/tabs were restored and no query history was recorded
I would expect that these part-executions are recorded as well. Even if I execute the “stage 3 output” only, this also is kind of a full aggregate up to this point - otherwise Studio wouldn’t be able to execute it in the first place. Please add this to history.
I mentioned it but wasn’t able to reproduce it reliably: Sometime Studio starts without my old session (and tabs etc). I feel like this happened after crashes as well after PC restarts as well. But it is like 1 in 10 and not consistent, so just a small FYI.
Hi Hannes, thanks for these! Can I ask if re. point 3, do you have Session Restore turned on?
Yes, I do, great feature! As said, 9 of 10 times Studio starts just fine, everything is still there. Just this 1 out of 10 when my last session is not restored and I don’t know why (yet)
I will keep an eye if I recognize any pattern when it’s not working.
Any chance you could pass on your log,txt (and also global_exception.log since you mentioned a crash) to either Yvonne or me?
We’ve had a few cases where Studio 3T assumes another instance is still running and disables Session Restore because of this, even though no other instance is running.
Sure, will do, I will send the zip with all logs to both of you.
I had a clean start yesterday or the day before again, maybe theres something in these logs explaining that. I’m not expecting any fix from you, just providing some input for further investigation.
It’s me, again 
Good things first:
Still waiting for history for stuff that happens in scripts though Now, some feedback:
Studio3T does not provide a simple config/tasks export if you want to quickly onboard a new Studio user with your setup. Connections export/import is good. I know that there is some kind of config folder deep withing your windows profile where everything is stored, like tasks and stuff - but a UI for this would be nice.
To use Edit Collection History you either turn it on in Preferences or you tick a checkbox in the Update dialog, which is not working for me. I see the checkbox but can not click it, nothing happens. I had to use the global preferences setting. Screen:
And once again, just like here (point 1), I was not able to find this History UI for a few minutes straight, although I already searched and found a year ago. Yeah, you say I’m just dumb (
), or it is hidden too well. At least put it into the right-click context menu of a collection, better also put it into some program menu, and/or context menu of a database or connection (where I can see all collection histories of that DB/connection altogether)
Just like in my very first post in here or here as well, we (my team and I) really dislike the yellow/grey (theme dependend) popup that occurs when you handle ObjectIds or Dates (maybe other types as well) in the Visual Query Editor. And there are cases where you can’t get rid of it, it won’t close on click, which is even more annoying:
We don’t use the information within and it’s annoying the smaller your screen resolution is. I was forced to use a 1080p laptop with no other screen for some time and this popup blocked one third of the available height of the visual query editor. Nah, please, add a preference for this. 
Another Visual Query Editor: When you have a small screen and a complex visual query, you often have to scroll up or down within this UI part. If your mouse is hovering over a dropdown the moment you scroll, you will scroll the dropdown items down/up, not the UI. This is really disturbing because you can easily fuck up your query if you’re not cautious. Also scrolling dropdown… why on earth would you want this?! I would be interested if anyone of your team preferrably uses dropdown this way instead of clicking it to drop down the options and choose the desired one Buuut I guess that this might be UI default behaviour of your UI design framework rather than your devs coding it in. But maybe you can disable/block this behaviour, would be helpful.
Aaaand another thing on small screens with Visual Query Builder (yeah, probably avoid using small screens…) 
: Your new feature to show “Where”, “And”, “Either”, “Or” Labels might be nice and helpful for IT-newcomers and n00bz in general (sorry 
) but I really know what $and and $or means and now have to live with 25% less horizontal space in my UI which also leads to multiple problems:
Hi Hannes, thank you so much for this. We really do appreciate it. I’ve passed this onto our team who work with updates and if we include any in a future release, I will let you know.
