If I recall the Verb-Noun idea is supposed to make it clear what is happening, take a look through stuff like the approved verbs for defining cmdlets. There’s aliases and stuff for sure for example I think ls is an aliases for Get-ChildItem in PowerShell.

It’s supposed to make it so you don’t necessarily need to look things up, need to do something to an item? Well you can Copy, Remove, Rename, Move etc, and while yeah that’s a super basic example that you know the equivalent linux commands for, the concept is supposed to apply everywhere. Now, whether or not people follow the guidelines is probably another story.

I don’t really hate shell scripting, feel like they all have their place, complex stuff though is nicer in straight PowerShell than bash IMO, but I’m fine using either.

Synapse link is a pain too if you’re doing everything with as much private networking as possible. Actual setup is quick, but you need a windows machine for the PowerShell libraries needed for the dynamics side of the link, and if you’re just added as a guest to a client tenant, the cmdlets won’t let you login on their tenant, always uses the default tenant as far as I recall and there’s no tenant flag. I’ve set it up a handful of times and once it’s up it works really well, just an annoyance sometimes getting there. Think doing it through event hub has some similar irritations too.

I’ve not had the pain of dealing with fabric extensively, most of the engineers and data scientists I work with hate working with it, everything seems like a halfbaked implementation of stuff in synapse, adf and Power BI premium but somehow worse, and their documentation is increasingly unhelpful.

Haven’t looked into it but do shops offer lube analysis services? Yeah you could send out your own sample to a lab, having it as a shop service would be way more accessible to people.

Though, in my experience, getting people to commit can be a pain, lots of “yeah I know we have a long p-f interval and it’s super noticeable before it functionally fails, but it’s not that much effort so I’m doing needless maintenance anyhow just in case”, which end of the day you do you.

Was my first experience with source control, a bunch of Gary’s Mod mods were distributed that way, think I recall wiretool doing that, spacebuild was for sure, predated my work use by like 5ish years.

I didn’t hate it but definitely prefer git, but I’ll take literally anything over not having it,

Considering that is nearly exactly some of the answers I’ve received during the technical part of interviews for jr data eng, you’re probably not far off.

Shit I’ve seen solutions done up that look like that, fighting the optimiser every step (amongst other things)

I worked in primary metals for a while, core business applications still ran on mainframes, they had a project to move some of them to SAP that apparently had the same timelines as nuclear fusion (perpetually x years away).

Thanks for that, Bugdom also looks familiar!

Mine had a bunch of iMac g3s, eMacs came toward grade 8.

Games weren’t explicitly forbidden, just needed to finish work first, new Cross Country Canada, math circus and Oregon trail were the games I recall the most of. There was this one game though I can’t recall the name of but the concept was interesting, you played as a time travelling velociraptor and had to save dinosaur eggs from extinction, was like a 3rd person shooter, I have no idea why that was on school computers

Edit: was Nanosaur

In the distant year of 4122, a dinosaur species, Nanosaurs, rule the Earth. Their civilization originated from a group of human scientists who experimented with genetic engineering. Their experimentation led them to resurrect the extinct dinosaur species; however, their victory was short-lived, as a disastrous plague brought the end of their civilization itself. The few dinosaurs resurrected were lent an unusual amount of intelligence from their human creators, leaving them to expand on their growing civilization. However, as the Nanosaurs were the only species on Earth, inbreeding was the only possible choice of reproduction. This method largely affected the intelligence of the various offspring, and slowly began to pose a threat to their once-intelligent society.

The Nanosaur government offers a quest that involves time traveling into the year 65 million BC, where the five eggs of ancient dinosaur species must be retrieved and placed in a time portal leading to the present year. Their high-ranking agent, a brown Deinonychus Nanosaur, is chosen to participate in this mission. On the day of her mission, she is teleported to the past via a time machine in a Nanosaur laboratory.

Yeah, been dealing with that a bunch lately too, I’ve started pushing them towards the documentation directly (though to be fair, sometimes that’s ass or nearly nonexistent) with some success.

Sqlbi for Power BI, Marco and Alberto have pretty much been my go to in that world for probably near a decade (though I haven’t really done a lot of that in a long while).

I tried it, immutable is not for me on the desktop, went back to arch. Bazzite had HDR working on the desktop with my nvidia card, ended up doing the same in arch after finding out it there was a flag I needed to set, personally haven’t had an issue with that set.

If you’re good with immutable though, it seemed decent enough to me, was little to no fussing to get things going. I don’t really distrohop though, historically I use debian on my machines but arch has been a solid experience in the past month.

Could use Polars, afaik it supports streaming from CSVs too, and frankly the syntax is so much nicer than pandas coming from spark land.

Do you need to persist? What are you doing with them? A really common pattern for analytics is landing those in something like Parquet, Delta, less frequently seen Avro or ORC and then working right off that. If they don’t change, it’s an option. 100 gigs of CSVs will take some time to write to a database depending on resources, tools, db flavour, tbf writing into a compressed format takes time too, but saves you managing databases (unless you want to, just presenting some alternates)

Could look at a document db, again, will take time to ingest and index, but definitely another tool, I’ve touched elastic and stood up mongo before, but Solr is around and built on top of lucene which I knew elastic was but apparently so is mongo.

Edit: searchable? I’d look into a document db, it’s quite literally what they’re meant for, all of those I mentioned are used for enterprise search.

Postgres runs well in a container in my experience and is nice to work with, def support that. I know sqlite works well, but nice to have a system sometimes.

Taco according to the cube rule

Seriously though, it’s been some time be afaik any microsoft product file that ends in x, .docx, .xlsx, .pbix are all just archives and you can totally interact with them programmatically if you want. Really easy to corrupt them but hey, found it interesting years ago.

I’m always personally wary of storing blobs in a database if for no other reason it’s going to totally be more expensive to store on a server rather than in some sort of blob storage.

Supposed to be an easy, if not a drop in replacement afaik, it’s under a permissive licence (Apache 2.0), beyond that it’s authored by RedHat I can’t tell you much else, it’s something I’ve been considering moving to personally (and work, pretty much for licencing and the few of us that want to use more open tech stacks) I just haven’t had a chance to work with it.

Supposedly able to pull docker images and work with docker-compose, just not swarm.

Yeah, had that happen a few years ago, thankfully there was a consistent status attributes in the response that I could use but still, annoying

A good chunk of it is relating to the elastic search stack, yeah it’s a thing people do.

I have a half thought that maybe bing works well for technical searches because it’s the default search engine for edge and depending on the company, you may or may not be able to use a different browser and I’ll be real, I tend to leave my work laptop setting as default as possible unless particularly awful.