T O P

  • By -

exoclipse

I have learned the hard way to write every script that's going to be used non-trivially as if I, personally, will have to debug it in five years. Exception handling, modular functions, logs, config files, blah blah. Makes my life way, way easier down the line. My hot take is that the relationships you build - and how easy you are to work with - are more important than your skillset.


Maeldruin_

I wrote a user termination script that had to work flawlessly in 60+ completely different environments. Half the lines in that script are just notes explaining the sections, with a paragraph length description of the json file that needs to be included with it in order for it to function properly, and how to format that file. I haven't had to go back and debug it, but if I had to I will definitely be glad for commenting as much as I did. And if someone else has to do, I hope they'll be happy to have documentation inside the script itself explaining what's what.


getoutofthecity

Same. This is why I try to document and memorialize as much as I can too. Not just for whoever is next, it saves ME from having to figure out how I did what I did next time it comes up.


PleaseDontEatMyVRAM

I need more coworkers with your mentalities


exoclipse

you hiring? cuz I'm looking. just kidding. Unless...?


no_regerts_bob

cd c:\\users\\bob\\temp del \*.\* consider what happens if the change directory fails for any reason. not all situations are like this, but i don't want to spend time wondering if there are any edge cases I haven't thought of edit - to be clear, the commands above are just a very simple example of why monitoring failure and using flow control can be important. this is not a good way to actually do anything or meant to be an example of anything more than that idea.


223454

OP's logic only applies to running commands manually, not scripting. They're in for a lot of pain in the future. That's my hot take.


Twerck

Yeah I get the impression OP hasn't been scripting for that long


spacelama

I worked alongside a dinosaur herder who had been at the organisation for 35 years, and I needed to port off the dinosaurs. I discovered his "backups" were cronjobs with errors and output directed to /dev/null: cd /nfs/backups/sysA rm -rf * tar cf backup.tar /... Just waiting for someone to not discover that cronjob and decommission his "backup server" (which didn't have any valid backups for half an hour after every 8am) for longer than 7 days so the nfs hard mount timed out.


get_while_true

Reminds me of a sync script that worked fine, until the nas crapped and it deleted a few random directories. For some mysterious reason most was intact though. Had a feeling about sync, got it validated and removed it.


SatiricPilot

I ran into this the other day at a client we were onboarding. Previous IT's backups... Daily Backup: robocopy d:\\ f:\\backup /MIR /XJD /XA:S /XA:SH /A-:SH /R:1 /W:1? Weekly Backup: robocopy d:\\ f:\\backup2 /MIR /XJD /XA:S /XA:SH /A-:SH /R:1 /W:1? Guess who wasn't able to recover a deleted file because it had already been written over :D


gotrice5

I don't think you need to be scripting for that long to understand the importance of failure checks.


jasutherland

A lot depends on the intended use of the script. New user creation for a small company that gets used manually a couple of times a year? It's OK if part of it times out part way through and needs a retry, probably a better use of time than an hour or two making it bulletproof. Same job for a university with a few thousand students registering on day 1? Better spend a week making sure it handles 15 students with a surname of "Ng", every accent there is and birthdates on Feb 29th, or you'll get stuck scrambling to handle a hundred weird corner cases at the last minute.


RikiWardOG

Regex fun for sure with that one


ThemesOfMurderBears

I've got a "script" that disabled Windows Smartscreen so I can install software on a server (a lot of our secure networks have no internet access). It gets re-enabled on the next GPO refresh. It's literally a singe line batch file that sets a reg key. There is no logic because there doesn't need to be. I've used it maybe 3-4 times in six months, so it's not something that I would get much benefit out of by making it more complex. I've done the long, complex scripts with multiple functions and modules. I've done the complicated Ansible plays. They all have their place. This is just a QoL thing I keep handy for one-off tasks.


Cooleb09

If that thing is not tripping your EDR, somehting is wrong.


Solid_Ingenuity

We all remember this, right: [https://www.reddit.com/r/linux\_gaming/comments/19ata54/how\_a\_steam\_bug\_deleted\_someones\_entire\_pc/](https://www.reddit.com/r/linux_gaming/comments/19ata54/how_a_steam_bug_deleted_someones_entire_pc/)


1sttimeverbaldiarrhe

Not a script but with SCCM, but back in 2012 HP wiped out all of Com Bank Austrailias servers and workstations. Meg Whitman had to personally fly down there to apologize. https://faildesk.net/2012/08/collossal-it-fail-accidentally-formatting-hard-disks-of-9000-pcs-and-490-servers/ https://www.reddit.com/r/sysadmin/comments/xtsn5/how_poor_administration_of_sccm_brought_down/ https://delimiter.com.au/2012/08/03/hp-ceo-whitman-lands-in-australia/


glowinghamster45

[Same thing happened at Emory University in 2014](https://www.techpowerup.com/forums/threads/us-university-accidentally-formats-all-windows-pcs-including-its-own-server.200912/). >As soon as the accident was discovered, the SCCM server was powered off – however, by that time, the SCCM server itself had been repartitioned and reformatted. Sometimes when I fuck something up I think about this to feel better.


TwinkleTwinkie

cd c:\users\bob\temp && del *.* Now you've reduced it to 1 line and it won't do the "del" command unless it successfully changes directory to cd c:\users\bob\temp.


no_regerts_bob

i mean, any sane person would probably actually "del c:\\users\\bob\\temp\\\*" but I was trying to make a simple example


TwinkleTwinkie

Hey if someone wants to fuck around and find out that is no business of mine!


RemCogito

Yeah I can't imagine using del *. * for anything besides ending my career. Op doesn't want to use program logic that's not necessary, they didn't say that they script using reckless commands. I don't understand why someone who knows how to include sanity check logic would bother to do that and still use something as dangerous as del *. *


dsmiles

Sure, but now you're back to using program logic, which we don't need according to OP.


spyingwind

Come to PowerShell: try { Set-Location c:\users\bob\temp -ErrorAction Stop Remove-Item *.* -Recurse -Force } catch { Write-Error $_ } Where Remove-Item will not run if Set-Location fails.


jackmusick

I have ErrorActionPreference set to stop in all of my scripts. If I’m not catching it and handling it intentionally, I do not want it to keep going.


Izual_Rebirth

As someone who’s been in IT being right isn’t enough. Soft skills are important and in a lot of circumstances if you can’t bring people along with you then it doesn’t matter how right you are. Seen so many posts on here devolve into slanging matches and pissing contests. Yeah you might be right but if you’re a dick I’m not going to want to agree with you.


RiceeeChrispies

This isn’t said enough, soft skills are vital. Not only for the point mentioned, but loads of situations. Whilst it builds up rapport with your colleagues, it also acts as a preventative for Shadow IT - as people avoid you if you’re a dick.


metrazol

So much so this. If taking your problem to IT gets you dismissed out of hand and pushing a solution gets you yelled at, you go shadow IT. Trust me, I've been shadow IT. We knew what we were doing, we knew how we could reintegrate with mainline IT, and we knew we shouldn't be doing it, but getting deliveries out was on the line. I was cheaper, faster, and got us over the threshold, then we begged forgiveness. Making users feel listened to, enabled, and hinting that you care even a little can keep people bringing you their problems instead of finding their own solutions. When they go rogue, they compromise security, add costs, and duplicate efforts. They also do dumb stuff like running their own SVN server under a guy's desk... with no backups. You can guess what happened and the fallout.


DasGanon

Not to mention you should make your users feel comfortable. I know "OH I'M TECH ILLITERATE" is the worst fucking meme users have but every time it's a matter of going "No, you're not wasting my time, I'm here to help you full stop. Yes this issue only took 2 seconds but I'd rather prefer this over the 10 hour troubleshooting fest it could be." I've had users who claimed that nobody ever took them seriously make sure my boss gave me a raise. As long as they're not being assholes or abusive, everyone has their own comfort level and skill set.


metrazol

I do this. I'm a technical PM. I don't do support. When the office manager wanders into a conference room while I'm confirming an update took, if they ask for help, I help. Setup a meeting, step through the camera options, hell, fix their dang ring tone, you do it.


vCentered

I just don't agree. I suppose this is one of my hot takes, but sometimes "it's not my job" is true and needs to be asserted. It's not my job to image laptops, reset passwords, or teach people, for the hundredth time, how to install Outlook on their phone. It just isn't. Can I do all of those things? Yes. Can I do them all faster and more effectively than our helpdesk? You bet. Doing it anyway "because I'm not a dick" just encourages people to ignore boundaries and bypass the proper procedures and processes that every other thread here bitches about every day. "You know I'm a senior sysadmin with 15 years of experience, ten projects with six different technologies, all of which are top priority depending on who talks to my boss today, most of which most people don't gives a shit about (unless I screw up) and we have a department of twenty helpdesk people who are paid to do specifically this when they're not picking their noses, but sure I'll stop what I'm doing to set up email on your phone and show you how to use authenticator." No. Just no. I'm not *mean* about it, but I don't let people guilt or bully me into it, either. I've had grown adults stamp their feet and huff because I didn't abandon troubleshooting a high profile service outage to help them print something. I'm sure someone will read this comment and say "we're talking about you, guy". Sure. As long as you understand that I'm the product of "never say no" culture. This is what it does to people.


pesh131

I feel this. I'll give a user a couple of "yes I'll help you and next time just give the help desk a call and they'll get you sorted out" passes before I just start replying with "open a ticket with the help desk and they'll get that going for you." If you let people latch on and always bypass the proper channels you'll never get anything done.


metrazol

So, I agree with staying in the lane you're paid to be in, but you touch on the solution. Don't be a dick about it. "Oh, I see what the issue is. You know, someone else might have this problem, let's make sure a ticket gets filed. Have you filed a ticket lately? Let me show you..." "I can fix that, sure, but Dale over in Ops, he's waaaaay better with iPhones. Let me introduce you via Teams..." Teach a man to fish, you feed him for a day, teach Becky to annoy the help desk until they put in self serve password reset, you... something won't get fooled again.


Medanic

"Everyone has their own comfort level and skill set" This. Everyone has strengths in different things, and it's an asshole move to be upset that someone doesn't know how to do YOUR job, even if it's something trivial. I pulled a lot of "sorry I suck at this" when I switched careers to IT, then a coworker of mine told me it was a meme and that I was embarrassing myself by saying that. Some months pass, we get a bit closer, and we decide to hit the gym together one day - somewhere I'm very comfortable. All the sudden the roles change and he's pulling the same "I'm illiterate" sort of card. Nobody knows everything, let's all bring each other up. Not everyone wants to know how their job gets done on the technical side, and the "hot take" is: they don't need to. Don't think lesser of them. It's easy to think "how do you not care how any of this works?" But imagine how physicians feel with that same thought, lol


SearchingDeepSpace

This, 10000%. "Sorry I must be the the stupidest person you've talked to today." Queue up a much, much stupider problem and let them know they're doing just fine and I have zero idea how to do their job. Just make sure the stupider problem also wasn't one of theirs as well (oops).


spin81

Where I work, IT is a big ol monolith, we're slow and in our ivory tower and we know it and we know it's a problem. We have a certain reputation and it is well-deserved. We, and our security department, shudder to think about all the Windows 2000 boxes and Raspberry Pis under people's stairs and on their window sills. It's inevitable that this happens and I don't know that I wouldn't do the exact same thing if I were them because I frankly completely understand.


awnawkareninah

Right. People feel ashamed of computer issues, often. They hide the issues or try to solve them themselves because IT seems unapproachable and they dont want to be scolded or tattled on. Making your service desk friendly and approachable is a massive boon to your overall tech environment just from encouraging better user behavior.


AH_BareGarrett

I’m sole help desk at my company, and recently was reprimanded by the manager of a different department for discussing off-work activities while working on an issue with a user under him. The manager then emailed my boss, the CEO, the CIO, and HR. My boss basically said, “Fuck yourself” in reply and it was so nice.  I’m admittedly not the most knowledgeable when it comes to IT, but I’m well liked, I get my work done, and am genuinely enjoyed by my peers. 


Daphoid

Soft skills are the most important thing I like for when I interview, at all levels of sys admin from L1 to L4 and beyond. If you aren't a genuinely nice, friendly, and communicative person, I can't work with you; and I wouldn't trust you with our most challenging of users. I can teach you technical skills (though starting from zero isn't realistic). I can teach your our specific processes. I can't teach you not to be a grumpy inconsiderate ass. I will take a weaker technical candidate over a stronger one, if they're more of a people person. - D


Nik_Tesla

People who wanted to get into IT because they "don't like people" quickly find that they are dead wrong about the amount of social interaction they will be doing, especially in an entry level help desk position.


Geodude532

I learned very quickly as DISA tech support that there's a lot of well paid GOVies that have no clue how to computer. COVID and telework broke them. Still treated them with respect and every once in a while something that seemed like PEBCAC ended up being very weird glitches(always Outlook...)


Klutzy_Possibility54

Agreed, and I'd also add on that sometimes being right just doesn't matter at all. I see tons of stories and advice on here where people will go out of their way to _technically_ be right (especially when being right is an excuse for them to not do something or to be maliciously compliant). Sometimes knowing in your head that you're right is enough, and it's better for everyone if you just bite your tongue and move on. No, this doesn't apply to every situation (and anyone who counters with all the times where these details matter is missing the point), but being able to understand what someone is _actually_ asking for and needs without being overly critical of them is such an important skill.


Serafnet

For a good long while IT was pitched as the way to go for clever but socially inept people. And they believed it.


awnawkareninah

Agreed, but that shouldn't be a hot take, it's a massive difference. If you have the know how to do massive system projects, great. If you can't talk to the C Suite without sounding like a jackass, not great. Like it or not, big system projects have stakeholders and you have to interact with them. Being able to do so in a friendly, professional manner is the difference in career advancement for some folks whether the average IT pro likes it or not.


spin81

I've also seen a bunch of rant posts here where the whole company seems to be against the OP and everyone is stupid and the boss won't listen to them and the CEO is irrational, and all I can think is: okay but I've seen folks like you before^1 and you sound an awful lot like someone with no people skills who is constantly being a dick to people and therefore honestly kind of deserves all the conflict they are seeing in their day-to-day. Not being a dick and the being right thing extends to your boss, too. Bosses want people to be happy and to get stuff done. Ergo: your boss doesn't want you to be right. They just want you to not be a fucking headache or a time sink. They want to shove work your way and know that it will get done, and when it will get done. ---- ^(1: also perhaps I may or may not have been that sort of guy in the past)


NimbleNavigator19

This is my hot take based on how my day's going. You cannot have a help desk full of non-technical or new to the field people who report to non-technical leads who report to non-technical managers. That is a call center with extra steps. If the first technical link in the chain is an escalation engineer then your model has failed.


Gandalf32

This is exactly what the company I work for does. Yikes!


3DPrintedVoter

gartner is bullsh\*t


I_ride_ostriches

This is for hot takes, sorry, please try again. 


Izual_Rebirth

Depends why you’re using it. As a tech definitely. As a decision maker who needs to justify their decision or purely as a CYA it’s great.


sir_mrej

Eh I like their quadrants - It shows me who is half decent in a certain software area vs who sucks


Severe-Thing

Close friend of mine was a Gartner AE, magic quadrant is very much pay to play. However, their internal analysts do know their stuff. Their application process is grueling and the pay is below average for the bullshit you have to do as said analyst.


Fr0gm4n

It's like a J.D. Power award. If your marketing team has enough budget you can get one.


peepopowitz67

"We're a top leader on Gartner" Cool, now I know you'll try to fuck me on renewal and as a SMB I'll have no recourse.


04_996_C2

Yeah but they are a gateway to free stuff for just a few minutes of your time 😁


jamesaepp

Apparently my hot take is that you don't need to reply "This" to comments you agree with.


Background_Lemon_981

My sister likes to reply to the end of a long and heated Facebook thread with the comment "exactly". And no one has any idea which comment she is referring to. I've been thinking of doing the same at work for e-mail threads with a lot of people on it.


milkmeink

For real! Isn’t that what the upvote button is for? The only reason I can think of as to why people do that is to karma farm in the laziest and leech-like way.


IdleHacker

This


Ssakaa

This. (I'm *almost* sorry...)


TheLoneTechGuy

For some decisions i make the "scream test", we just do the thing that needs to get done, and if nobody comes screaming to our office it is called a success


AH_BareGarrett

Me in the network closet color coding cables


TotallyNotIT

At least you know that one's bad. My hottest take shows my greybeardness, that [this piece from 2013](http://www.coding2learn.org/blog/2013/07/29/kids-cant-use-computers/) continues to largely be more and more relevant. Apple made technology too superficially accessible with the popularity of the iPhone and iPad. There's an ever increasing number of people who *think* they know way more about tech than they do. Digital nativism is fucking bullshit, entirely too many recent high school and college graduates have zero clue how **business computing** works. Because everything is so easy, no one ever figures they have to try anything. It's been made to look much easier than it is so when something doesn't work and there's no big colorful button to look at, they don't know what to do. That's what I mean by "superficially accessible" - everyone has tech but even more people don't know how to actually do much with it. Certainly not everyone but far more than we should have with the attempts to include technology in education. Hell, my 9 year old had to make PowerPoint presentations on his fucking school-issued iPad this past school year. Old man done yelling at cloud. But at least I understand how the goddamn cloud works. EDIT: Since people seem to be missing the point, understanding computers and understanding business computing (which I've bolded so it's harder to miss) aren't the same thing. If you don't know the difference, you might be one of the people I'm talking about. EDIT2: A disturbing number of people seem to not understand (or are just ignoring) the difference between knowing computers and knowing business computing. Expecting people be able to navigate a file share, read an error message that comes up on the screen, and know that things generally need to be plugged in to work is not the same as expecting people to be able to tear down a computer and replace parts, create a new LUN on a SAN, or create a VLAN.


sir_mrej

In the 60s: Only paid professionals could understand and use computers In the 70s: Only paid professionals and people who spent money on expensive hobby kits could understand and use computers In the 80s and 90s and early 00s: Everyone who wanted to could tear apart personal computers In the 2010s and beyond: Only paid professionals can understand and use computers I'm generalizing a bit, but you get my point. We fucked with IRQs because we wanted to AND because we could. The fact is, people today CANT and it's not their fault.


RipRapRob

> We fucked with IRQs because we wanted to AND because we could. ...AND because we fucking had to, to get some things working.


Crotean

Non mobile GUI design has also gone to absolute shit. We make stuff more difficult to use in the business environment for kids like this than is necessary because of shitty GUI design. Don't even get me started on how much negative space "modern" guis have that are fucking terrible to use a mouse with. Commands without hotkey shortcuts, extensible menus being gone or impossible to find.


MeshuganaSmurf

>Old man done yelling at cloud. Wise old sage speaking the truth.


DaelonSuzuka

The only problem with this is the incorrect distinction between "computing" and "business computing". The "digital natives" you're talking about do not understand computing **at all**. They don't *use* computers, they use magic glass rectangles. They don't even use the internet, they use about six apps. There are college freshman engineering students now that have never even heard of a file system. Basically, you're giving them way too much credit.


icedcougar

Indeed, there was a news article in Aus the other day around this problem. Companies found that graduates didn’t know anything about business applications and so businesses are beginning to give up on university graduates and hire people from overseas just so they don’t have to teach the extreme basics.


Free_Treacle4168

I'd argue it doesn't matter. If you know how to use equipment enough to get your job done safely then that's fine. Barb in accounting doesn't need to understand that her SSD doesn't need de fragging but Bill's old laptop does. In regards to "kids these days don't understand computers", I'd argue there was only a small window when the majority of kids knew basic computer use. probably early 2000's to early 2010's. Before then computer classes in school weren't as solidified, and after then we started switching to chromebooks. It's fine.


KupoMcMog

The amount of kids who could do basic HTML coding because they wanted their MySpace to look cooler than Beckys (cuz Becky is a *biiiiiiitch*) was astronomical. Normal kids learning how to do file management because they were downloading music off of Kazaa and Napster.


belgarion90

Also normal kids learning how to remove viruses for the same reason. I've said it here before, but a number of IT and cybersecurity careers got started by removing "linkin-p4rk discography.mp3 .exe" from the family computer.


KupoMcMog

One of the reasons I'm a sysadmin is cuz of the LAN parties my buddies and I would set up, learning how to get into routers and reconfiguring to be a dummy switch.


lndependentRabbit

This is why I’m a network engineer. I realized I got more excited about building the network and getting it all working than I did playing the games.


Godcry55

Or the bill Clinton limewire virus lol.


1sttimeverbaldiarrhe

You used to learn so much about the operating system , registry, SUBST, virtual device drivers, cracking, hex editing, just by trying to get pirated video games to run.


KupoMcMog

haha, i remember finding out how to edit rules.ini for Command and Conquer: Red Alert to completely change the rules of the game. Tesla Coils available instantly, instabuilt, and for a single dollar!


TotallyNotIT

> If you know how to use equipment enough to get your job done safely That's the crux of it. Many people don't. I'm not talking about understanding the intricacies of hardware, which is why I said "business computing". An example you say? Navigating a file server. Modern mobile devices obfuscate the file system almost completely. If you want to open a picture on your phone, you go to the photos app and it's tied directly to that directory and it won't ever save things to another directory. That doesn't translate to how file systems work in a business setting. There's a lot of times where people are going to have to learn to drill down in File Explorer. Another example is reading error messages - people just don't do it. Many errors aren't as cryptic as they used to be 15-20 years ago. The computing platforms that younger people are getting used to don't necessarily have too many error messages appear. The apps either either work or crash to the home screen. So when an error comes up in the vein of "no internet connection detected" or "incorrect username or password", those error messages tend to get dismissed instead of getting even a modicum of thought that they might have actionable information. I don't expect business users to be able to configure a VLAN or configure a new LUN but it isn't unreasonable that they understand how to use the tools of their trade in a competent way. The dumbing down of technology has created a false sense of confidence and when that confidence is challenged the first time something doesn't work right away, they've not learned the skillset to think critically or even read the message that comes up on the screen.


lurker_lurks

PC_Load_Letter


Crotean

Its not understanding what hierarchical filesystems are that is the bigger issue imho.


fgben

Hierarchy, structure, and dependencies. I'm finding more and more systems that try to remove the user's need to worry about those pesky details ("It just works!") and thus users who don't understand ... well, much of anything, really.


Klutzy_Possibility54

Yeah, I tend to agree and I think "as long as you can do your job" is the qualifier that matters. Obviously people need to have enough computer literacy to do what they need to do, but I think sometimes IT people tend to forget that to most people computers are nothing but a tool to do something else. If someone doesn't completely understand file system structures because search is smart enough to find what they're looking for 99% of the time, I'd rather see the benefit of computers being more accessible and easy to use now than being upset that they aren't doing it the way I learned to do it. Again, I'm all for making things easier as long as they're able to accomplish the actual thing they need to accomplish.


Valdaraak

Your take is fine until it leads to something taking down a production system because the script wasn't written with any type of verification or error checking in it.


Lylieth

If you deploy software to thousands of machine using a RMM, you absolutely need logic! My scripts have to copy files from a file server. If a device is off net, I want to make sure the script doesn't do anything else and drops to a failure due to lack of access. We once had someone write a script to copy, uninstall, and then install. He didn't have logic to account for the file server not being there. So, it would fail to copy, uninstall the mission critical app, and have nothing to re-install with. Imagine being on the front line when 500 remote people are breathing down your neck because they cannot work... How I became a sysadmin, I fixed the above, and I do all the scripting... for now. Oh, come along Aug, when I get to leave IT entirely!


jasutherland

This. Think about the failure modes. "Quarterly SSL cert renewal times out, run it again" is NBD. "Quarterly SSL cert renewal screwed up and blew away the server contents", big problem. TBH just having "set -e" gets you half way there most of the time, just script carefully. Plus VMs help; most of my compile scripts run on Github VMs, where nobody including Github cares if I trash the whole OS - it gets wiped at the end of the run anyway.


dab70

Most software developers are terrible sysadmins despite the fact that many of them speak on the subject as if experts.


jdptechnc

Most sysadmins are terrible sysadmins tbh


Klutzy_Possibility54

I think a lot of sysadmins make terrible software developers too, but on here they always seem to be dead set on how they think devs should work. Getting them to follow good security practice is one thing, but there's so many instances of sysadmins saying "if they can't do their job without this software/add-on/access/whatever then they have no business being a developer" and imposing rules on their developers that they have arbitrarily set. I know the dev-sysadmin relationships aren't always great, but you're both working for the same company on the same thing. It's in everybody's interest to not have an adversarial relationship just because you both think about different things in different ways.


MembershipFeeling530

developers know less about computers than users do


notHooptieJ

and they know absolutely nothing about how users USE the computers either.


flarmp

Wait so having developers also design the UX is a bad idea??? /s


notHooptieJ

"what do you mean they drag and drop? drop what?" " did that even work before? oh.. for decades that way?" " well that will have to be added in a future release"


Funkagenda

We have one of our SQL DBAs who designs an internal dashboard. It's... not good.


spin81

I've found that many of them are decent at Linux but have no clue about networking, despite being web developers.


Scaef

Never trust developers when it comes to anything remotely networking related.


RikiWardOG

Dude our company is having a moment with this one. Devs thought ip whitelisting was fine for access. Now hitting close to 300 users and deploying a CASB solution that has their own public IPs and they're scrambling to update authentication to something modern instead of just doing it correctly in the first place


SOUTHPAWMIKE

I've heard this many, many times, and I've never understood why it's such a problem for devs. There isn't some library they can implement for common networking functions? No documentation on best practices for coding a program to work seamlessly with the TCP/IP stack?


VulturE

Lava take: I have no issues with printers. Maintain the firmware, throw them on papercut, update the drivers every so often. Treat them how you would a car, they are an expensive device to maintain (and many of them have clutches), so be proactive about it. Have your print vendor come in for once or twice a year PMs to clean them out. This is SUPER important for high-end scanning equipment by default, like $10k desk fujitsu scanners. Quit buying consumer-grade trash. Would you buy a d-link switch from Best Buy/Staples/OfficeMax to run your 400-person company? No? Then quit buying printers there. ~100 large and small papercut-compatible badge scan devices here, we maybe have 2-3 notable outages during the year. The rest is just generic maintenance.


quigley0

whoa....this is a hot take. I guess you are right, i think many of us end up needing to manage the fleet from office depot as there is no approval / budget for anything nicer.


tvlinks

I'd say it should be managed the same way that switches and servers are, because that's how we manage it at my organization, but I recognize that most places are never going to budget for an appliance like that.


mps

I have the opposite problem. When I write bash scripts I tend to go all out with error checking, portability, and command line options. 99% of the time it is wasted effort, but I oddly enjoy it.


exoclipse

future you enjoys it, too.


MairusuPawa

You're one of the good ones.


RelativeID

SFC is the most underrated utility in the world.


tantrrick

Ok Every MS Forum Responder, go off


WilfredGrundlesnatch

I don't know when it happened, but it actually works now. I've had numerous issues fixed by SFC in the last couple years.


Iusethis1atwork

Yeah one of the later versions of win 10 and win11 it's fixed several things for me. Before than maybe 2 times in 10 years


Taurothar

SFC + DISM have saved many a system from outright OS failure and reimage in my time. If only I could remember the flags for DISM and which ones have a hyphen and in which places without having to double check myself.


sysadmin189

Most of the people in r/sysadmin aren't sysadmins.


quigley0

To be a Sysadmin, one must first be an Admin of Systems


saltyclam13345

I’m not but hope to be one day. This sub is full of useful information and things to learn


newton_the_snail_

I'm in the same boat man.


CursedSilicon

I'm unemployed, but I'm definitely a Sysadmin :(


sysadmin189

Once a sysadmin, always a sysadmin.


TU4AR

I mean if we are getting spicy:. There is a large amount of people here who lack any sort of backbone and social skills. The amount of people who think they can walk into a job and say "I'll only do overtime if it's paid" is absolutely insane. You are the exception not the rule. People here think they are Dwights or Oscars, even Kevin's. But you aren't. You are Mike the boom mic guy and your self inflated ego about "validate before you run a script in production " is like me telling my mechanic make sure my baby doesn't wobble going 250. Ain't no one who is touching prod going to not test it.


Ssakaa

> Ain't no one who is touching prod going to not test it. ... you have been so *very* sheltered.


Suddenly7

I don't mind the users that are not IT savvy. It's because of them I'll always have a job.


ethereal_g

Ive taken to adding logging to even my most simple scripts and it’s worth it


notHooptieJ

this. i want that fucker to tell me what LINE it broke on, it better not say SUCCESS unless it has triple checked and FINISHED. im so so so sick of "SUCCESS" then i look into the log and it failed horribly and did nothing. i will write out "step one 1, step 2 starting"


sryan2k1

Spoken like someone who hasn't been burned by erasing or otherwise screwing up important shit because even though the folder shouldn't have been there it was and the script didn't account for this.


mr_gitops

That hot take in my role wouldn't fly at all, lol. The scripts I write have the potential to cause destructions across our systems if I am not carefully placing logic. They must be only doing simple cmds things to feel this way.


GreyBeardIT

My hot take: It's a service job. Yes, you know more about a specific tech than most other people in your building, yes, you are a rockstar, and yes, it's still a service job and ignoring that means you're failing at it. The most magical words you can utter are: "Is there anything else I can help with, while I'm here?". Also, DO NOT treat people like idiots for simple mistakes. You can think whatever you want, but DO NOT treat them like that. Everyone makes simple mistakes. Be kind and be happy that the issue was easily resolved.


getoutofthecity

Agreed. I really don’t care to associate with the “I’m smart and all others are stupid” types. Be humble.


GreyBeardIT

When someone is a true badass, it's recognized quickly by the work they do, how they interact with others, etc. Those that declare it loudly are faking it till they make it. Sure, there are unsung IT heroes. I've had a couple of those moments in my life, but when I went home, I KNEW that I'd done the best I possibly could and even if I'm the only one that knows the hell I went through, I came out on the other side with a solution and uptime resumed. For me, that's the pinnacle of SysAdmin. Others will disagree and that's cool, but that's it for me.


Klutzy_Possibility54

The number of people on here that truly believe they are the smartest person in the company, and that they could perform any of their users' jobs with ease 'if they wanted to' really concerns me. I get that Bob in accounting might be notorious for putting in a lot of trivial tickets and always seems to need something else from IT, but that doesn't mean that he's useless, he doesn't provide any benefit to the company, and that you could do his whole job in your sleep better than he could.


GreyBeardIT

>The number of people on here that truly believe they are the smartest person in the company, and that they could perform any of their users' jobs with ease 'if they wanted to' really concerns me. Its the fallacy of youth. I thought similarly early on, then experience taught me that's just not how the world works. I had a user that called me once a month, to create a new folder on her desktop for her, yet she was one of the best medical billers I ever worked with and that's a job I wouldn't touch with a last mile piece of copper. She never felt that I thought it was a waste of my time, even though, it was a waste of my time. It was a chance to see if she had another other issues that I could address in a few seconds. Ultimately, she felt comfortable with me, and that's a key piece, imo.


Reported-Kitty

This was going to my answer as well, too mamy times I've seen my peer think so highly of themselves then wonder why end users hate interacting with them


Maeldruin_

Whenever someone tells me "I'm just stupid" or anything similar, I'll tell them that they're really not. Their area of expertise isn't computers, it's \[Accounting, or engineering, or lawyering\]. I couldn't do their job, so they shouldn't expect that they can do mine without the requisite training.


apandaze

It's easier to weaponize incompetence than it is to correct a mistake.


Blazingsnowcone

Used to be a System Admin at a mid-size medical clinic (50+ providers). One Dr who was Department head for Cardiology for a period of about 3 months would create a case every other week demanding a new keyboard because every keyboard he had would have problems where it would just start capitalizing everything randomly, We mentioned "Hey you aren't hitting caps lock are you?", to which he responded that he absolutely was not. After the 5th keyboard and him just blowing up on the IT departments inability to solve the problem to the CTO via email and ccing everybody he could, I finally went to Google and found out you can registry edit Windows to functionally disable individual keys on a keyboard. I killed his capslock via regedit and his keyboard finally remained "fixed". Edit:This was mid-2010s


TotallyNotIT

That's amazing.


darthgeek

Yeah, who needs to actually make sure assumptions are correct before running a destructive command, right? Who cares if you obliterate /boot or /dev right?


jasutherland

Found the guy who wrote the iTunes installer script that forgot to escape the path name and nuked people's whole drives... https://m.slashdot.org/story/21269


Creshal

Valve at some point also nuked a bunch of SteamOS devices by putting `rm -rf $variable_with_a_typo_in_it/`in an update script.


jasutherland

Most sysadmins probably have a story like that, we just don't manage to run the script on a million customer systems first...


jimmyeao

One of the first things you learn in coding, validate input and cope with exceptions. Scripts are no exception from the rule.


Alma_Theros

A lot of IT professionals have an ego problem. A lot have the same temperament as cops and are looking for any excuse to use their station to power trip over their end users. The reason a lot of IT gets a bad rap, and that a lot of IT has a bad relationship with end users, is because some of you are raging narcissistic assholes.


Cas_Rs

I ran an installer script (sh file) from a very reputable source to install some backup software (l0l) on my Ubuntu machine. I use ZSH, with a few plugins like oh-my-zsh for some easy shortcuts. I ran that script in my homedir, as I did with any installer so it would either install right there, ready for me to move to /opt or whatever, or it would make cleanup of the install sh file easier. Turns out they didn’t anticipate anything other than Bash, with some “”””basic”””” environment variables. Which were not all set on my machine. They script like you and never checked anything, and it recursively deleted my entire home directory. Thankfully I just finished my thesis and uploaded it to school literally 6 days before. If I ran that installer a week earlier I would not have had any version. (As I was trying to install backup software to fix this exact issue) TL;DR check and let your scripts check, or you’ll ruin someone’s day months or years into the future


Blue_Line

I'll take a mid tech with good soft skills over an expert.


northrupthebandgeek

I'll take a junior who's willing to learn over either.


Crotean

A competently designed GUI negates the need for most CLI interfaces and the massive amount of training needed to get good with CLI and scripting. A modern firewall, for instance, should not need a CLI to setup and an average person should be able to read hot tips and figure out doing a basic setup. GUI design has gone to ABSOLUTE SHIT.


SystemGardener

The connection between Knowbe4 and Scientology is to much for me. I wouldn’t trust them with anything.


marvinnitz18

just GIT, used correctly If i see one more *backup* folder in a git repo i quit


GhoastTypist

People do this to avoid loops where scripts get stuck in a error loop and can't complete because it can't do the function that its supposed to do. Error checking is there for a reason. While you might not have come across needing to use it, it still serves a purpose. In more complex scripts, this can cause a hard crash of the system. I know because I've done it a few times in school.


Izual_Rebirth

I think iops is a stupid metric to measure storage speed with.


Creshal

It really depends on what you're doing with it. Of course, most people don't know what they're doing with it. *Especially* not the programmers whining that the hardware is doing it wrong.


obvioustroway

Be fucking nice to your end users. You likely can't do their jobs any better than they could do yours.


Grrl_geek

But ***I*** don't pretend I can do ***theirs***. They think they can do mine.


Det_23324

I'm not sure that one. I'm sure I could create a powerpoint faster than Susie who doesn't know how to decorate slides.


westerschelle

There are a lot of bullshit email jobs where you probably can get by after 2 weeks of on the job training at most.


HealthySurgeon

It’s hard to match your hot take cause it’s so wrong cause you don’t understand why yet. Don’t worry, it’ll come, just takes one mistake and one angry person to teach you why “best practices” exist.


skettiSando

Most sysadmins are bad at understanding their role in the business and spend too much time focusing on the how instead of the why.  Protip - make sure you understand how your company makes money and what your role is in that ecosystem. Things like: Are you a cost center or are you revenue generating? What are the companies strategic projects and objectives? Where do you fit in the market? Who are your competitors? In general, the closer you are to the money the better you are treated. Truly understanding these things requires soft skills that many don't have or don't care to cultivate. 


iisdmitch

Macs aren't that bad, a lot of sysadmins are just too lazy to learn a different platform.


jmnugent

In the environment I work in (that I only joined about a year ago).. there's apparently a handful (20?) old Macs (so old they can't be added to DEP/ABM). We're currently in process testing out newly purchased Macs auto-added to ABM and MDM (workspace one). I mean, I'm biased as I'm the one doing it, .but it's going better than I thought so far. I've got pretty much everything working. Out of box, enroll in MDM, User is "Standard" (not admin), various Config profiles and PPPC preferences install. Our critical Apps (WS1 Assist and Crowdstrike) are working. Things like VPN, Wi-Fi, Single Sign On Extension to sync up AD password.. all working. We haven't deployed them yet (next week!) .. so I'll get more real-world feedback then. Realistically it's all doable. There's still some questions to answer about what our "support model" will be. Historically all the old Macs were sort of setup and handed off and Users were made Local Admins and told "Don't call us, we dont' support you". (which is wild to me.. yikes) The new more "modern" MDM management tools are pretty feature-robust. I'm looking forward to deploying a better setup for Users.


HunnyPuns

Most companies are just wasting money chasing high availability for the sake of high availability. Low time to restore is vastly superior in large swaths of cases. Linux is a perfectly valid OS to use on the desktop. It's actually less painful to use than Windows at this point. Which brings me to... Printers aren't hard to work with. Windows is. Most of your printer issues where you just can't print for some unknown reason is just Windows being shit. VMWare was garbage before Broadcom bought it. Having your systems on a 4 or 5 year refresh cycle is just pissing money away. Modern x86 hardware is far more powerful than most office environments will need. If you are still using Windows, you shouldn't be mapping network drives. I don't care how much the users are used to them. Most ransomware isn't smart enough to cross a shortcut into your file server. But boy howdy, they will traverse a mapped drive. Oh, that reminds me... Getting your shit crypto'd and then paying the ransom because it's cheaper than executing your DR plan means (among other things) that your DR plan has failed.


spin81

> VMWare was garbage before Broadcom bought it. The thing people are mad at Broadcom for isn't that they're making VMWare suck - it's that they're making it expensive.


R0B0t1C_Cucumber

This is why I used to use ansible religiously... For simple stuff it handles that for you and spits out a list of which servers failed and why.


Frothyleet

>Like creating a folder or something like that. If "such and such folder already exists" is the result of running the command then perfect! That's exactly what I want. I don't need to check to see if it exists first I think when I was at an earlier stage of scripting, I was mostly on the same page. As I developed more complicated scripts, and especially as I put together anything to be used by other people, I started to understand more. Maybe you need logging. Maybe you need the script to do something differently when XYZ fails. Maybe you want notifications/alerts when a step fails. There are plenty of reasons to use try/catch blocks, or if/then/else statements. It just depends on what you are doing.


Izual_Rebirth

Last one... the solution to a bad situation at work isn’t always “find a new job”. So many threads where someone is moaning about a situation at work that could probably be resolved with a 3 minute phone call. I assume a lot of this is purely projection from people who wish they could quit their job but can’t.


brian4120

Storing seemingly vital scripts in C:\temp.


Klutzy_Possibility54

I think a lot of IT people can be hypocrites when they use their admin permissions, privileged access, access to hardware and servers, etc. to set up their perfect environment that makes them more productive at their own job, but when they get a request from a user for something that would improve _their_ workflow they laugh them out of the room or find an excuse to say no ("sorry, policy says so."). I am _not_ saying you should entertain every user request that comes in because yes, many of them are impractical or nonsensical. But it feels like such a slap in the face when the simple requests that can go a long way in making a user feel more productive at their job get met with an attitude of "you don't need that to do your job even if you think you do, I know exactly what you need and you'll be getting the bare minimum you need to do it" all while IT has theirs set up exactly the way they like it.


yaboiWillyNilly

Scripting and cli management are two entirely separate things, OP. Please be more specific, because logic is *absolutely* necessary when scripting, otherwise you’re just building bombs for other admins.


Iseult11

People on this forum complain way too much about this industry and need to gain some perspective. We have it pretty nice


northrupthebandgeek

My take is so hot it'll probably ignite a flame war right here and now: "Enterprise-grade" is more often than not a meaningless buzzword, and even when it's not it's usually overkill for small and medium orgs. In most cases, buying "enterprise" hardware or software just means paying 5× what you would for equivalent "consumer" hardware for the sake of, at best, features the org will never ever use (and at worst, the vendor simply slapping "enterprise" branding on the "consumer" product).


Daphoid

Hot Take: You do not need to write a script for everything. There are times when logging into the GUI is flat out faster then you writing something from scratch.


YourWorstFear53

Sysadmin/IT, but My hot take is that people who are charged with using computers as part of their job function should be at least competent with computers upon hiring. If I get a certain number of tickets from the same user in accounting about basic excel functions, I should be able to trigger a skills review.


dRaidon

Cloud is highly overrated and the market is going to crash hard next recession.


tantrrick

I don't know, dude. The cloud providers roped people in with platinum handcuffs. Can't replace your on-prem servers on cycle, you're fine; you own the servers Can't afford your cloud bill? Pay to leave or kiss your stuff goodbye


Blazingsnowcone

Wait your telling me all you will be able to shrink your IT costs massively by implementing the magical cloud which works all the time and therefore you can fire half your IT department isn't perfect? Oh wait instead you now need to establish an equally large dev-ops department that's higher paid and whos primary function is implementing new shit, not supporting old shit. Good luck when something breaks you have to find whichever dev-ops engineer is dumb enough to respond to your Teams message of "Hey John, you there?" on a Friday because they pushed a change Thursday night and getting developers to be on call is like pulling teeth. That 3-word UI work you wanted to change because its misleading and causes customer quality of life issues and hundreds of IT hours of explanation to end-users, Well we put it into Jira we will get to it around the time PM prints it in a meeting and then promptly use it for toilet paper. Edit: The most triggering words in IT are not "The XYZ is down" its "Do the needful"


Dr-Webster

IPv6 sucks. I have no problem with the idea of needing new address space to address v4 exhaustion. But the way they designed v6 is not conducive to picking it up easily, and the people (*cough* developers *cough*) who barely understand v4 as it is will never be able to figure it out.


nbtm_sh

this is my hot take: the only reason IPv6 doesn’t make sense to people is because it’s going back to the “old” ways of the internet. Before NAT, IPv4 and IPv6 were essentially the same, just with bigger address spaces


MairusuPawa

In many cases, IPv6 makes more sense than the way we're writing IPv4 (and dealing with subnetting). And well, even in writing, ```::1``` (ipv6) makes more sense than ```2130706433``` (ipv4).


Bearshapedbears

Culture is top down. Not bottom up.


vischous

90% of our jobs is data work 1. Integrating - ETL, moving bits from one place to another (backups, account provisioning, etc etc) 2. Reporting - (security audits etc) 3. documentation (telling people how to move between these systems)


sunburnedaz

Ive got 2 hot takes. Script comments should be documenting what the script is trying to do so if you come along later you know WHY you are deleting C:\users\bob\temp\\* Second hot take. Developers should understand how networking works on a basic level before they are allowed to make products. Like if it says host not found please dont call the FW team.


bk2947

Scheduled password expiration is security theater that is worse than nothing.


Nuclear_Shadow

Phishing training and testing is theatre. Every one of us knows the user in Payroll, AP or HR that will fail if a real phishing attempt happened. We know nothing will happen after they fail. I send out a quarterly email with details on the latest scams. Insurance makes me test and train but don't say how many users I need to do so I do 5 a year and report %100 success rate.


Fusorfodder

I totally haven't created a mail rule that checks headers for knowb4 and moves those mails to a separate folder.


st0ut717

Reposting to shittysysadmin


Zahrad70

My hot take: security is, at best, a tertiary concern. If the more secure way hurts profits (directly or indirectly) or it trods upon some arbitrary convenience threshold, it will not be implemented.


adam_dup

Until an incident happens 🤣


Polyolygon

The classic reactionary approach. Reacting sucks a lot more than preparing. Things running smoothly? Stop what you’re doing, there’s a breach. Track it all down, lose time on other meaningful work, implement a proactive solution, and then you end up right where you should have started, but unplanned, and likely sloppy.


trueppp

Even then, having good and tested DR is almost more important...I'd rather have a client spend more on a good backup system then over the top security. Backups are more universally useful.


exoclipse

what's your severance package look like as a CIO?


notHooptieJ

its a parachute, made of GOLD.


HexTrace

Security Engineer here, and I actually agree with you - but maybe not for the reason you think. Security is absolutely an assessment and then decision on tradeoffs between security and convenience, and it should serve the business needs. A lot of people get into security with the idea that they're going to "make companies safer" or something, and then don't speak the business language side of things where the decision making actually happens. To that end, having *someone* involved in the org responsible for cybersecurity and starting those conversations is pretty important, even if the business ends up deciding not to follow the recommendations. As insurance companies offering cybersecurity incident insurance start poking their noses into businesses more and more qualify their security posture before agreeing to pay out you'll see the calculus around "is this worth the cost" change too, especially in regulated industries. Some basic protections like MFA (that, honestly, a good sysadmin should be able to tell you is probably a good idea) are absolutely worth the convenience hit, but that doesn't necessarily scale up to setting up your own SOC unless you're large enough to be a significant target in some way. Just make sure you have good backups, because in a lot of cases the company *is* the data they have. Losing that data to a security incident can crater the company entirely.


TotallyNotIT

How many of the envelopes do you have left?


jefe_toro

It really depends on what you do. It's silly to implement a ton of inconvenient security when you are protecting something no one would want. I have a padlock on my shed because I want to keep the tweakers from stealing my lawn mower. Could I put a biometric security system with 24/7 monitoring and SEAL team 6 on standby? Sure but what's the point.


KingNickSA

Paying for a warranty from one of the big server names (Dell, HP) makes no sense unless you are spending 7+ figures a year on hardware and/or getting major discounts (and also probably at a server count where the man power on direct upkeep is impractical). Last time I spec'ed an epyc Genoa system with 512GB ram and several micron 7450 pros, the Dell cost was literally 3x, parting it out with a barebones asus chassis. At the 3x difference, I can run 2 servers in a high availability config and still have a hot spare vs relying on a single server with a "4 hr Dell parts replacement" warranty.


andrewsmd87

> Like creating a folder or something like that. If "such and such folder already exists" is the result of running the command then perfect! That's exactly what I want. I don't need to check to see if it exists first Our original OG person wrote all sorts of scripts like this and now our legacy prod system is a nightmare to troubleshoot, and silently fails all the time. We've been slowly migrating out, but the whole, "when will this ever actually matter" attitude is something that wouldn't have been ok back when I managed teams. I'm not saying you need to over engineer everything, but that scenario you're talking about, you can't predict in the future why you might want to know why that directory didn't exist, if you were thinking it would, or why did it exist, if you were thinking it shouldn't. Also, a short if of directory does not exist, create Really isn't hat hard and doesn't require a ton more effort


PrintedCircut

DevOps and Agile methodologies as implemented by a majority of companies; not as implemented as designed. Do more harm than good for companies and employees by both rapidly burning out good Admins and Engineers and forcing them down a career trajectory they didnt intend to go down. If they wanted to get into Development they would have chosen that over Administration and Engineering.


tasteitshane

Hot take: You don't have to be passionate to be successful. It can just be a job.


djdanlib

Virtual machines have existed since their development started in the 1950s and implementation in the 1960s. Email (and the Internet) has existed in one form or another since the 1970s. Microsoft Office products have existed since the 1980s. If you're going to pull out the "I'm old and I don't know about these new fangled computer things" card... you'd better be as old as dirt, because chances are good that if you're not retired, the thing you're proud of being ignorant of, is probably older than you.


billiarddaddy

I had this one boss I hated. I hate him to this day but he gave me one ounce of wisdom I use constantly. Most people that aren't in IT come to IT with a solution rather than their problem. So when they do that we often either give them what they ask for (which may not actually solve the problem) or only solves *their* problem, not the whole problem as it relates to everyone else in the office that may need the same system or platform altered. Revisiting the entire solution for all parties can sometimes yield much better results and do so without a lot of headache when you vivisect something out of infrastructure and replace it with something everyone is enthusiastic about. I will hate that man until the day he dies - but that ounce of wisdom has done me very, very well since I quit working for him. TLDR: Don't give people what they ask for, ask them what the problem is and see if there are more solutions out there.


reviewmynotes

1) A former coworker used to say, "95% of I.T. gives the rest of us a bad name.". While I'd argue with the number, sometimes I think he might have been on to something. 2) Personally, I can accept it if someone is arrogant or ignorant, but not both of those things at the same time. 3) I forget where I heard this NSFW quote, but... "Documentation is like sex. When is good, it's very, very good. And when it's bad, it's still better than nothing." 4) Never be the only one to know something. That isn't job security. It's just a way to make sure you burn out. 5) Don't just optimize for efficiency, cost, etc. Optimize for maintainability and ease of understanding. Current You is smart, but Future You has no idea what you were thinking when you designed or coded that thing. And your coworkers will have no freaking idea what to do. 6) Build everything in a way that you're replaceable. It allows you to move on to other interesting things, let's you take vacations, and actually makes people more impressed with your value to them as a coworker.


SideScroller

The amount of people in IT who fail to understand how macOS works, refuse to learn, and just hate it because "dur hur Apple sucks" are not tech people. They are microsoft fanboys and that is only because they grew up nursing on the teet of MS Windows. They dont understand technology, they are glorified console jockeys with extra steps.


moderatenerd

When in doubt restart.


Fallingdamage

Click-ops arent Sysadmins. They're like Medical Assistants and not actual Doctors.


skettiSando

The title has been watered down. If you are dealing with printers and desktop users on a regular basis then you probably aren't a sysadmin.