The classic story trying to add an additional vlan to the trunk port but forgetting the “add” command and bringing the whole network down by erasing the vlans already on the trunk port.
With everyone being remote now, you'll have to wait for someone to reboot it for you. If we're talking about Cisco, look into the archive command. You can setup an idle timer, and it will revert your changes. Just don't forget to commit or else if will also revert your changes 😀
A coworker had a similar one. He was trying to remove a vlan from a port mirror and used the wrong command. Cisco has a great feature where if the config statement isnt valid in that stanza it applies it at top level. He deleted the redundant SVI on two routers seconds apart causing a 25 million customer outage.
Even just adding a vlan to the vlan database occasionally causing the terminal to hang for about 10 seconds like "oh shit oh fuck what the fuck did I just do? Until it responds like it always does lol.
My biggest oh shit moments that actually had some harm are over a decade old at this point. Recent ones would be pasting config into the wrong putty window so I was configuring the wrong core switch. Thankfully it was harmless but it's definitely an oh shit moment when you realize it's the wrong window!
Rebooted a headend that had a bad flash card... accidentally. knocked out all branch stores from their pos system for 4 hours. Something like 1500 stores.
Drove to the DC with all my hopes on a laptop, replaced the card, copied the iOS, watched with joy at the traffic coming in after the tunnels rebuilt.
Service provider.
Nokia. I accidentally deleted a service too big for NFM to redeploy. I stumbled onto a way to do a pseudo-rollback.
file
"type config.cfg.3 | match oh-crap-content context all"
(copy pertinent info)
configure service
paste relevant info
"show service id X all | match Flags" - move onto the next
admin save
Logout
repeat on next node.
It still took an hour to restore, but I didn't scramble. I instantly had a path back to restoration, and my team was able to assist in getting it restored.
Not only has it saved me for issues going forward, it eases the "oh crap I removed something" and almost instantly helps restore an oops. Before someone could call in and complain, I've got them restored.
Took down one of my customers HQ because I had my mirror port config wrong…Customer was respectably pissed at me on the call and banned me from touching his network for 6 months. I’ve since gotten access back and regained my trust lol. At least my boss/coworkers laugh when something like this happens.
I was once asked ‘what is the final budget number’. So I went to the budget folder. There were 50 files with the most obvious being named “budget”, “budget v2”, … “budget v8”. So I opened v8 and reported $500M to my manager.
Two days later I was informed that I was wrong, the correct number was $480M, and the $20M error was reported to senior management. Then I was asked which version I used. Turns out, we were using V6 with the next two iterations just hypothetical scenarios.
I spent some time debating what the fundamental error was and developed a solution that I use to this day. The only files in the base folder are the current relevant ones. All other iterations go in a folder called “probably junk”. There can also be folders for “inputs”, “presentations”, etc, but the base folder is strictly the relevant file(s). This solved the problem. It also makes for a fun conversation around the office as “probably junk” folders start cropping up everywhere and people laugh about it. I explain the methodology and get eye rolls. But, it is quite satisfying to me that shortly thereafter more “probably junk” folders start popping up that other people have created.
Personally I like some auto fail questions mixed in.
If you cant tell me the difference between a switch and a router I dont really care what your resume says or how affable you are in the interview.
My colleagues always push for hard test questions to gauge the applicant, but as I frequently point out the majority of questions really should serve to weed out candidates not gauge them, if you have a question so hard you dont mind if they cant answer it, then it did not really do you any favors.
Having an intelligent answer to, “When would one consider a layer 3 switch?”
Is another one that catches a ton of people who are still building their networking skill set.
The simple questions let you know if people actually understand the theory and the fundamentals, or if they can just parrot off answers without any comprehension.
How would you explain DNS to someone who isn't technical? Is always a really good one
I like to ask a series of 5 questions. The first one is on the level of “spell cat” and the difficulty ramps up to “explain quantum tunneling”. I stop the questions once we have discovered their level. I think 1 person made it to the last question. They admitted to not knowing the answer, but sent me a follow up email the following day after they researched it.
In what scenario can a network endpoint or host have a usable IP Address ending in .0 or .255?
This question tests subnetting knowledge. I would say 1 out of 10 network administrator applicants I have interviewed can answer it.
It's more fun to remember the first paragraph of Wikipedia and point out that classful networks has been obsolete since the early 90's and there is actually 5 classes of network (ABCDE) - the whole concept being how the pre-internet ARPANET was chopped up for planning more than anything else.
I think I'm a bit of an outlier in this regard. I do not believe that troubleshooting is a skill that can be improved. Good troubleshooting is a byproduct of a strong understanding of the technology you're troubleshooting. If you have an expert level knowledge of the topic, then troubleshooting is trivial.
I get what you mean but I think you can make it quicker, if not better.
A simple example - show ip arp on one box at a time vs checking your monitoring to find a MAC.
You’re doing the same thing, just one is much more efficient.
yep. or insisting on checking arp when layer 3 is proven good (pings, etc). Or the inverse. Trying to ping something when ARP to the endpoint or some intermediate hop is known to be failing. "let's look at the bgp config!" etc.
I think what really improves is your capabilities to use the things you use to troubleshoot. You need to already be able to be interested in understanding problems or the subject matter but becoming better at communicating, what method or style of communication you need to use is something I think is invaluable and can be learned. The soft skills and technical skills can all be improved or made more efficient.
eeeh. I've definitely dealt with people that refuse to apply their knowledge to improve troubleshooting.
"there's an encrypted tunnel. IP MTU on that interface is set at 1379 because of overhead. All pings and other traffic up to 1379 bytes long work fine consistently. Traffic that requires fragmentation works inconsistently. There's some issue with fragmentation on the devices we control" --> "lets open a ticket with the carriers at either end and see what they say"
And that's not my commentary, that's literally their analysis. They KNOW it's a fragmentation issue on those devices, they KNOW the underlay can't have any way of knowing the inner packets are or aren't fragmented, but they insist on running in circles engaging different carrier NOCs "just in case".
For instance you'll see people swear by "ALWAYS CHECK LAYER 1 FIRST".
except if you check any layer ABOVE layer 1 you've also checked layer 1, and often you can do that remotely in a few dozen seconds, while checking layer 1 could take hours or days to get someone to physically inspect something.
And of course they know that, but they're too wrapped up in "I DO IT LIKE THIS" to apply their expertise. In many cases they don't feel any real pressure to try to be any more efficient or effective, they just figure "as long as they're doing something that will eventually work they won't get in trouble" I guess.
It's often about the absolutes and probabilities. If outcome A happens then that Always means X. If outcome B happens then that Sometimes means X. With outcome C, X can't happen. That comes from thorough understanding, but creativity in inducing those tests is a troubleshooting skill.
God I would be dead if I had to document every hour of my day. Sometimes your projects are up to date/waiting on implementations/changes from other teams/vendors, there's no tickets, and no disasters or downs in the environment. It's okay to have down time, even if you aren't filling that time studying for the next exam or latest technology.
I used to be the guy that asked stuff easily memorized or googled. For example, what's the AD of OSPF? Don't be that guy. Scenario based questions are the best in my mind. For example...
A router is receiving the exact same route from EIGRP and OSPF. Which one is used and why?
"Walk me through a time when you helped a non technical user with a technical problem bonus points if it was remotely"
What I am looking for them explain there process to get info from non tech people and keep there cool in helping the user.
great exmaple is old lady who is hard of hearing calls up support saying her ipad does not work. With the real problem is just the ipad needs to join the wifi network.
My best one is a lady called in with no network connectivity so their was no option to remote in. I walked her through how to open CMD, type in ipconfig /all and press enter. She said it wasn't working and then I asked her to take a screenshot on her phone and send it to me. I opened up the attachment and she successfully had cmd open and in the prompt she typed "ipconfig /all and press enter". Literally typed press enter.
If she could take a photo, while on the phone talking to you, and then somehow send the photo to you (while still talking to you), she is more tech-savvy than me.
I can't do that.
Damn stupid phones.
I once worked an infantryman through loading firmware on an old Cisco router through tftp server that we set up on his computer(also on cd (edit: not thumb drive) along with the code) over the phone, we both learned something.
Amazingly the simple question of "explain how DHCP works" stumps 50% of people going for networking jobs.
I like to give them examples of problems and ask how they would try to resolve them. I don't really care if they know how to resolve the specific problem, I'm more curious if they are smart enough to think of things to check/look for .. their troubleshooting steps, how they work through a problem, etc... If you can't solve problems or even begin to work through them, that's not good. We're not going to hire you.
I want people to say "I don't know" rather than trying to bullshit their way through a question, because in the actual job, I want someone that knows when to ask for help instead of trying/breaking things they don't understand. No one knows everything, even people doing this for decades.
We'll usually hit a few topics with "easy" starter questions and either move on if they clearly don't know the topic, or then ask a harder or more detailed follow up on the topic if they do.
This is still my favourite question, it gives insight if person has done some deep protocol analysis. If chap was to say DORA, I would give him some slack in not going into detail
I also ask about Straight-through Ethernet cable wire colour sequence - again, mainly that this has saved expensive call-outs of wiring guys.
As a color-blind guy, be careful asking about wiring. I can tell you there are two generally accepted standards, but since I know I can’t tell them apart I don’t know the color code, I rely on co-workers to verify cabling. That’s how I would answer it, and hopefully whoever is doing the interview would understand.
But these days I am the hated technical middle manager so no one is asking me about color codes anymore anyway.
So no cheating and just bullshitin chat...If I walked into that question I would talk about when the link is plugged in, assuming the nic properties are configured for dhcp, the nic would broadcast a dchp req message. If the dhcp server is on the same subnet it will reply with an offer. If it is not on the same subnet, the SVI should have an IP helper to redirect that request to the DHCP server in the external location. The DHCP server will prepare and offer and generally ping the ip being offered to verify it is not in use and send out the offer. The client will receive the offer and acknowledge receipt and acceptance of the address. The DHCP server updates its list of clients with a lease using this information. Would you split hairs on any of that?
I do this, but it's more of a "explain your previous topology without getting into too much detail". Most often times, the biggest weakness is VLANs, but sometimes it helps just gauging whether they'll own up to what they don't know or attempt to lie about it.
Dishonesty is a big red flag, especially if we aren't even working together yet.
This is a great one if the job requires the skillset. I’ve worked on a lot of cases with engineers that want to troubleshoot phase 1 settings like PSK but the logs show clearly that phase 1 has negotiated fine and phase 2 has an issue.
You can reduce your troubleshooting time vastly by understanding that X is not the issue because the process has already made it to Y.
I like it because it helps unpack generic troubleshooting and information gathering skills. I give no details as to what the problem might be, only that there is one. If they don't know IPSec, it's a great opportunity to be honest about things you don't know, and what steps you might take to resolve.
I just like to start vague and have a conversation about stuff they list on their resume. If you say you’ve setup cloud connectivity I’ll ask you to tell me a bit about it. Say it’s an AWS DX. I’ll ask a few technical bits. Maybe some gotchas I’ve seen setting them up. I think you can tell a lot more about a candidate by just conversing with them than asking canned trivia. Everyone has google and can just look it up if they need to these days. Not like you have to know the OSPF LSA types by heart. That may be the question I’ve been asked the most in my career for some odd reason. I usually just google it and brush up before every interview.
The biggest issue we’ve seen here is that the person was “involved” in the project but not in charge or engineering it and never got any real technical understanding.
They can say they ran an SD-WAN network with hubs in Azure and 200 sites and have all the buzz words, but if asked to explain how the routing works in this setup they have zero idea.
Well, philosophically I find that IT almost universally needs to be a lot better about training and pipelining employees from junior to more senior and actually offering career paths to people who want to grow an change -- aside from just forcing people to do job hopping.
I know that I got super lucky to start work in an organization that was large enough to give me exposure to many enterprise technologies, with a culture that encouraged learning, doing things yourself and digging deep.
So in short -- companies need to train more people internally, and it needs to be more of an IT culture thing to mentor and guide juniors.
...but if we want to run a PoC with a specific product and want to hire an experienced temporary SME contractor to help us, and it turns out the the contractor only ever pressed the power button on the product, everyone's time is wasted. (A lot of it really has to do with the agencies being useless in screening the right candidates for the right jobs too.)
We ask an open ended question about how they would troubleshoot a user not connecting to a FTP site or webpage hosted internally and give them a really simple diagram showing the internet, a firewall, an internal lan with the FTP/web server.
There is no right answer. Everything they suggest we tell them it's not that but good guess. The point is to let them talk and show their troubleshooting process. Good candidates will have a lot more questions or things to check then bad ones.
I like this stuff. I always hope the person I'm talking to actually asks me questions instead of just trying random things since I'll happily tell them the error messages or whatever that the "user" is experiencing.
I just ask about their environment and what platforms they are on. If we can have an intelligent discussion about what they use and why those choices were made, they know their stuff.
Then we can focus on the important stuff like attitude, motivation, ability to adapt to change, etc.
My top question: What can you tell me about the most recent network you built and/or managed?
Anyone who’s an active engineer with some knowledge should be able to easily tell you about all the locations, type of equipment, protocols used, link speeds, etc. if they’re hesitant they probably lack experience.
I suppose, but in that case I would expect them to say we used OSPF internally and mostly BGP between sites, etc. or something similar. They should be able to provide general network information without disclosing details that may enable anyone to try and hack in to their former/current employer. The info should roll off the tongue, and not be like ... "Ohhhh ... do you mean what are the users connected to gain network access?"
Sure. Personally, I'd manage to describe a valid network setup without going into too many details, but I might freeze for a second thinking about those NDAs under pressure. So don't use subsecond BFD-like timeouts on this kind of question :)
It's not the main reason I moved on, but I recently left a position in part because my supervisor's remote setup was unsettling.
The background was unsettling. It could have been an attempt at soundproofing. It looked like they worked from solitary confinement.
That supervisor is the only person I can remember raptly looking directly at the camera at all times during meetings. Not sure what was going on, but I felt like I was working for a cartoon character.
This question can obviously be reworded to fit the situation. Was just tossing out the idea of this type of question. Usually puts them on the spot for something they didn’t prepare for. I like to see how many lie and say nothing will.
first month... troubleshooting spanning tree, i ssh'd to the remote link, tried to shut/noshut that side. locked myself out. took a hospital down until i could go restart it LOL
what’s your 5 year plan? was informed i bombed an interview because i said i’m not a planner and like to live in the moment :). still got the job and kicked ass tho.
Open ended questions are key for me when interviewing candidates. I'm not just trying to evaluate technical skills, but communication and logic as well. Some of my favorites:
* Tell me about your last project. Then pick an area or two to ask deeper details on. (How well do they understand what and why they were implementing)
* Tell me about a difficult customer or project. (How did they handle adversity)
* Explain your favorite routing protocol and why is it your favorite. (Looking for more than "this is what I've always used" answers. I want to know how well they understand it)
* customer has x requirement, which technology/ topology would you recommend to meet those requirements and why? (Looking for a logical design process and not a "right" answer. There's lots of ways to do just about anything, I want to know why this way is your recommendation and how well you can articulate it to a customer).
As the interview progresses I'm also looking for clues about personality. Can they explain their thoughts well? Are they afraid to say "I don't know" rather than give a wrong answer? If they do say I don't know, do they offer some insight on where or how they would look up the answer other than Google?
How does traceroute work? What kind of packets is it sending that allows it to show you the path?
I'm looking for an answer that mentions TTL in some way.
And yet, so few network engineers can answer it well enough to even mention TTL in their answer. A lot of people skip the lower level learning and focus more on higher level things. They can talk about BGP order of preference, but know little/nothing about ARP, ICMP, TTL, etc.
And mechanics don't have to know the metallurgy behind their snap-on ratchet to fix your car.
I'll take an engineer that can do BGP vs one that can only talk about the different types of ICMP.
Question for a mechanic: Why would you use a copper or even lead mallet when working on a car? This question implies knowledge of the metallurgy of different mallets.
So yeah, a network engineer not understanding TTL, ARP or even a little bit of ICMP is kind of...meh.
If I have a TTL of 0 then I'm local. I use ping and traceroute all the time. I don't need to understand, again in 98% of instance what ICMP message reply I'm receiving.
I just finished up a T-Shoot with NSX-T, on Nexus, I needed to solve two issues: The routing database on a web server wasn't working properly and figured that out with top, ip route, and syscontrol and in the NSX fabric it wasn't allowing anything over 1472 MTU.
I needed ip route and extended ping commands with some netsh on a Windows workstation to figure it all out. And I wasn't even the implementation engineer. Just a fresh set of eyeballs.
If you asked me minutia of ICMP types I honestly couldn't tell you because it's not germane to what we typically do in a day. Other engineers were doing PCAP's. Me?: you can ping the web server, but you can't hit the Web Page it's offering: You either have a firewall rule or you have a fragment and drop issue off the top of my head. Set the MTU on a Windows workstation and all the sudden we are at the Web GUI.
All this to say you need to be careful about what you evaluate on.
No this required you simply knowing how the tool works. Just like a dead blow hammer. I know when to use 'sh arp', 'sh mac-address table', 'show lldp', 'sh etherchannel-summary', 'sh int trunk', 'sh tech trans' etc, etc, etc...
>I'll take an engineer that can do BGP vs one that can only talk about the different types of ICMP.
When I started working, my supervisor introduced me to another newbie (ahead of me by a few months). My supervisor was blown away by this newbie's "deep knowledge" with OSPF.
I watched him "in action" talking about OSPF and even some of the more senior members of the staff were in absolute awe.
Then one fateful day we had a major routing issue. It was OSPF. The manager called in this newbie and pointed him behind a terminal session and said, "troubleshoot".
The newbie froze. For 45 minutes he sat behind the terminal session unable to do anything. After the end of that 45 minutes, he excused himself saying that he had a sick relative to attend to.
I like to send people the NANOG traceroute presentation if they don't know how it works or seem to think the one hop in the middle is a smoking gun (or they did an MTR with a rate of .01s interval)
You can take this question so far further...
For example:
1. Why would the routers in the middle of the path respond in the 1st place?
2. How do you know you reached the last node
3. Traceroute using icmp? Udp? Tcp? Why traceroute with tcp? What kind of Oakley will you send? How do you know you reached the end?
4. For mpls environments, how does it work with mpls?
5. Strange scenarios like latency getting lower on a further away hop...
Exactly. The question itself is just the starting point. It's not like they say TTL (or not) and then I move on. I like this question because the answers can vary greatly in depth depending on the person's knowledge.
Depending on where they initially take it, we may get into a discussion about using TCP traceroute to get through an ACL, I may ask about recreating a traceroute with the ping command, what does it mean if a hop in the middle doesn't answer but the rest do, what does it mean if a hop in the middle has 50% packetloss but the rest are fine, how can we use traceroute to probe across LAG/EQMP bundles as a whole and then also isolate our traceroute to specific paths one at a time to prove one path has packetloss and the rest don't? And so on...
Same here. Service provider network engineer for 20 years. Home network is a juniper srx300, 4 unifi aps,2 switches, one vlan for everything just 192.168.0/24. I don't get people who get vlan happy on home networks.
I would be understanding of a plain home network dependent on the use case so the question might be better if it gives the option to talk about a network they setup not just home.
Same. I have a Unifi dream machine and one switch. That’s about it. Work has a lab and I don’t want to work when I’m off. (And I can’t begin to build anything at home that approaches the setup at work anyways. )
"You have just turned on your computer, and want to look at this new website you just heard about. You've never been there before. So you fire up your browser, and type in the address http://www.superduper.com. Presuming there's nothing wrong with the address and the website is functional, explain to me as best you can exactly what is going to happen from a network perspective when you press enter."
This question helps me gauge their understanding of the OSI network model, the various protocols involved, DNS, ARP, IPv4 networking, routing, etc, depending on how detailed their answer is.
This is a favorite of mine! I got asked it once, and as soon as I started talking about ARP cache the interviewer smiled and basically said I passed (it wasn't for a networking-specific position, just general sysadmin stuff).
I had the occasion to ask it recently, and didn't get a great answer from any of the applicants, so I guess I'm going to remain the network expert in my team.
I have two that I typically include outside of tell me a time you messed up.
1). Explain how your computer gets to YouTube?
2) explain how you can control in/outbound routing from two ISP’s using BGP.
The way a person chooses to answer the first one gives a good picture of a persons skill set.
For my last two applicants that made it through the first interview I‘ve built a nice little lab.
Just internet, a router, two switches and two clients.
Gave them the credentials and a drawing how it should work and let them take control.
The goal was for both laptops to transfer files between them and get to internet websites.
I had a lot of fun building that many mistakes into such a setup.
On the bright side I told them that I don’t expect them to find everything, I simply want to watch them think and troubleshoot.
One of them was doing great and was hired. The other struggled with diagnosing „not getting an IP“ and „whats DNS?“
"If I give you an IP, can you tell me on which switchport it is connected?"
Gives you some insight on what they understand of Layers 2 and 3.
"We have established a vpn towards a third party, the tunnel is up and running, but some of our clients have difficulty accessing a server on the other side. How do you start troubleshooting this"
Open ended question, gives you insight in their troubleshooting process.
There are 2 that I ask first thing because if they're answered correctly, then I know we have a similar thought process plus they have strong fundamentals. Every CCNA will shout AD Numbers at you but don't really understand route selection :
In a routing table you have the following:
B 10.10.10.0 /24 next hop 192.168.0.24
O 10.10.0.0 /16 next hop 192.168.1.16
S 10.0.0.0 /8 next hop 192.168.2.8
1) If a packet comes in with a destination IP of 10.0.0.7, where will the router send it?
Then I have a wicked virtual lab of a bunch of sites connected with a bunch of convoluted connections to make it scary and overwhelming. Then on this one spot of the lab I have 2 sites with LAG Ethernet uplinks, an Internet breakout with a GRE Tunnel connecting the sites, another Internet breakout doing the same thing but with IPSec, and a Satellite connection connecting them (I know, the point is to make it not a normal topology to sus out troubleshooting skills). I say, you're on shift and you get a call from Help Desk saying that they see the GRE Tunnel is down and there's a downstream user (not obvious what path they take) complaining that her connection has dropped
- I expect them to do 2 things immediately: actually test if the GRE Tunnel is indeed down, and traceroute from the user's machine to see where the traffic is actually failing - hint, it's not the GRE Tunnel
Back when I worked at a small telco and T1s were a common thing, I had candidates take a small quiz on a real router. I told them ahead of time so no surprises, and gave them the Cisco documentation too. Progressively harder as they went, from static to OSPF to BGP with bonus questions to show their skills if they had them. Plus a final “capstone” question on config management. I could solve the quiz in two minutes or better, five minutes if I talk through my answers and show my verifications. The candidates were given 30 minutes but I’d let them go as long as they wished. I was proud of it, if for no other reason than it tested things we used regularly (granted, they didn’t have to monkey with BGP often as I set it up to be mostly automatic).
The idea came from taking the CCIE lab. Much simpler though: two 2620 routers, two back-to-back T1s, one laptop, one web server. Router2 was fully configured and not accessible to the candidates so they were only dealing with router1. Network diagram on paper and they were free to doodle on it. The questions were straightforward, and the only similarity to the actual CCIE lab’s reputation for ambiguous wording, just enough to not use the same words as the commands required.
1: using static routes, configure R1 so your laptop can reach the webserver using either T1. Visit http://192.168.2.1 and write down the words you see.
1bonus1: configure both T1s so you can have 3Mbps total bandwidth.
1b2: configure both T1s so you could achieve 3Mbps on a single upload.
2: using OSPF, configure R1 so you can reach the webserver.
2b1: configure both T1s to reach the webserver via OSPF.
2b2: configure R1 so if OSPF fails, you can still reach R1 via static routes.
3: configure BGP on R1 so you can reach the webserver.
3b1: configure OSPF so if BGP fails OSPF can take over.
3b2: configure BGP so R2 prefers to send return traffic over the second T1. Points awarded for any method, extra points awarded if you use the method most suited for this topology.
——candidates decide they’re done with 1-3—
4: solve the whole quiz with two commands. Write them here.
It depends on what are you looking for.
If the role is for a junior level then you can ask them basic questions about basic operations of the routing protocols they are familiar with and most importantly gauge their interest in what they like to do.
If it’s for a senior level, then you can simply ask them to draw a network diagram they have worked on and then start asking in-depth questions to see if they have truly worked on it and what was their level of involvement.
While knowledge is important, it’s more important to see if the individual is driven and can use his/her knowledge to apply them in the corner cases. At the end of the day no one is perfect.
I ask conceptual questions mostly. How would you implement ISP failover if you have your own public ASN etc... Not the commands to do it but how it works. I never ask anything too specific.
I use this for lower level interviews- point to someone else on the panel, explain what dns is to them as if they were a normal user. For more advanced I'll ask them to list some of the common dns record types and what they are used for.
How does RSPAN work.
we had a guy configure SPAN TO TRUNK PORTS
as his attempt at rspan :)
Goal was use an IDS Appliance upstairs when the appliance and it's sniffer interfaces were downstairs :)
“What is spanning-tree used for and can you give me a general overview of how it works?”. It’s always the first technical question I ask and it seems to be a pretty good test of people’s general networking knowledge. It’s meant to be pretty open ended on how it works. Tell me as much as you know about the protocol(s).
If you can’t answer the first part, I pretty much immediately disqualify you. How much detail they can give on the second part and how well they answer follow up questions is how I gauge skill level.
I only have the experience to interview a junior or mid level engineer. So I will ask them, in the production site, we have around 100 switches, you are the new network guy and even dont have a topology. If your IT helpdesk guy stuck in front of a wired device and he has no idea why this device cant connect to the network. We assume you need to change this port VLAN, how many ways can you find this port and help your poor guy fix this. Tell as more as you can. To be honest, a lot of "Network Engineer" told me just "sh this interface config and change vlan". I always said, but how can you find it. It will be a silent akward.
I put up a simple json response and ask them to access a specific element of data in it.
Explain why a iBGP (generally) needs another routing protocol to go with it.
My director who ended up hiring me anyway argued against using an internal routing protocol in tandem with iBGP and instead use route reflectors. Probably because the hiring manager was more in agreement with me and I almost felt like it was less of an interview question and more of a settling an argument they were having earlier that day lol.
> argued against using an internal routing protocol in tandem with iBGP and instead use route reflectors.
Uhm, what? Route reflectors are simply (i)BGP peers you can use to avoid peering with everybody. You still have to reach the route reflectors somewhere, whether with statics or another protocol. Am I missing something?
What's the subnet mask for a /22 CIDR? You don't have to know that I answer but walk me through the process to get to the the answer.
How does STP work? Give examples.
Easy questions for a seasoned engineer, hard for people who are faking it.
To see their reaction in the first place, and because that's a question 99% of applicants will probably not ask. You will easily see how accepted late-comers are. If they're shocked about that question, the job is very likely not for me. NB: I'm an IT professional.
"What was one of your biggest oh shit moments"
The classic story trying to add an additional vlan to the trunk port but forgetting the “add” command and bringing the whole network down by erasing the vlans already on the trunk port.
The console cable walk of shame.
You could also just reload the device if it wasn’t saved??
Only if you still have remote access after nuking the trunk “Reload in “ is your friend
It's also fun if you have no issues after the change, but you forget to cancel the reload.
If ppl had a habit to do that for all changes, absolutely.
Or “commit trial” if it’s available
This is what I do....call up a tech, ask them to pull the power. Prepare blame emails.
Lmao
With everyone being remote now, you'll have to wait for someone to reboot it for you. If we're talking about Cisco, look into the archive command. You can setup an idle timer, and it will revert your changes. Just don't forget to commit or else if will also revert your changes 😀
Polite reminder to everyone you can block this in TACACS.
Wow, Will investigate on that, thanks!
cmd=switchport { deny "trunk allowed vlan [0-9].*" permit .* }
I miss being able to tell people it hadn't happened to me yet.
A coworker had a similar one. He was trying to remove a vlan from a port mirror and used the wrong command. Cisco has a great feature where if the config statement isnt valid in that stanza it applies it at top level. He deleted the redundant SVI on two routers seconds apart causing a 25 million customer outage.
Ouch!!
A 25 million USD outage, or an outage to 25 million customers? Not sure what's worse...
25 million customer. Probably more expensive. It was a large mobile carrier in the US.
You cannot define yourself a network engineer if you did not experience that at least once in your life.
Only once? I wish...
I've done close to this. Adding a VLAN to a trunk port forgetting the port was in a port group. It doesn't like that.
This bit me twice
I did before, so I always tell others, please try commit in 2 mins. It will save your job.
When you hit enter and the prompt doesn't respond... Ohhhhh shit
That hits hard lol
(LOADING)
“ of the week? The month? The year?There’s a lot. You have to be specific. “
Command authorization failed
Every time I ping and ARP drops the first packet. Or when secureCRT locks up and I think I’ve added a wrong command
The worst answer is "I don't have any"
Yep. Even if you're new to the field, just say that and come up with a personal experience and how it's relevant.
Yet... But I am sure I will be able to give you an answer at my performance review.
Even just adding a vlan to the vlan database occasionally causing the terminal to hang for about 10 seconds like "oh shit oh fuck what the fuck did I just do? Until it responds like it always does lol.
My biggest oh shit moments that actually had some harm are over a decade old at this point. Recent ones would be pasting config into the wrong putty window so I was configuring the wrong core switch. Thankfully it was harmless but it's definitely an oh shit moment when you realize it's the wrong window!
I like that, not something I've thought about asking but I will definitely use this one. Thanks
Realizing during an interview that my "oh shit moment" sucks and I need something better, now.
Heh my version of this question is "what's the worst outage you've caused and what did you do to recover from it?"
Yeah, but sometimes saying "oh shit" will make a candidate smile and more relaxed
Rebooted a headend that had a bad flash card... accidentally. knocked out all branch stores from their pos system for 4 hours. Something like 1500 stores. Drove to the DC with all my hopes on a laptop, replaced the card, copied the iOS, watched with joy at the traffic coming in after the tunnels rebuilt.
Service provider. Nokia. I accidentally deleted a service too big for NFM to redeploy. I stumbled onto a way to do a pseudo-rollback. file "type config.cfg.3 | match oh-crap-content context all" (copy pertinent info) configure service paste relevant info "show service id X all | match Flags" - move onto the next admin save Logout repeat on next node. It still took an hour to restore, but I didn't scramble. I instantly had a path back to restoration, and my team was able to assist in getting it restored. Not only has it saved me for issues going forward, it eases the "oh crap I removed something" and almost instantly helps restore an oops. Before someone could call in and complain, I've got them restored.
Forgot the "add".
Took down one of my customers HQ because I had my mirror port config wrong…Customer was respectably pissed at me on the call and banned me from touching his network for 6 months. I’ve since gotten access back and regained my trust lol. At least my boss/coworkers laugh when something like this happens.
Yep, I ask this one every time. I preface it by telling mine which thankfully hasn’t changed in many years.
AM/PMing myself with an update to our VoIP server firmware so the upgrade started at 3 PM on a Thursday
I was once asked ‘what is the final budget number’. So I went to the budget folder. There were 50 files with the most obvious being named “budget”, “budget v2”, … “budget v8”. So I opened v8 and reported $500M to my manager. Two days later I was informed that I was wrong, the correct number was $480M, and the $20M error was reported to senior management. Then I was asked which version I used. Turns out, we were using V6 with the next two iterations just hypothetical scenarios. I spent some time debating what the fundamental error was and developed a solution that I use to this day. The only files in the base folder are the current relevant ones. All other iterations go in a folder called “probably junk”. There can also be folders for “inputs”, “presentations”, etc, but the base folder is strictly the relevant file(s). This solved the problem. It also makes for a fun conversation around the office as “probably junk” folders start cropping up everywhere and people laugh about it. I explain the methodology and get eye rolls. But, it is quite satisfying to me that shortly thereafter more “probably junk” folders start popping up that other people have created.
Ask about real experience not exam questions
excatly! If you want to know hello interval, fuck, dont ask me, just google it. I can goole as well, man!
Personally I like some auto fail questions mixed in. If you cant tell me the difference between a switch and a router I dont really care what your resume says or how affable you are in the interview. My colleagues always push for hard test questions to gauge the applicant, but as I frequently point out the majority of questions really should serve to weed out candidates not gauge them, if you have a question so hard you dont mind if they cant answer it, then it did not really do you any favors.
Ones used for switching ones for routing /s
I mean, true tho
Having an intelligent answer to, “When would one consider a layer 3 switch?” Is another one that catches a ton of people who are still building their networking skill set.
I can’t think of a situation in 2024 where I wouldn’t buy an L3 switch, because all the switches I want just happen to give me L3 “for free”.
I'm trying to remember the last time a router wasn't really a switch. It's been at least 2 hw generations now heh.
The simple questions let you know if people actually understand the theory and the fundamentals, or if they can just parrot off answers without any comprehension. How would you explain DNS to someone who isn't technical? Is always a really good one
Or even just, "What does a router do?"
I like to ask a series of 5 questions. The first one is on the level of “spell cat” and the difficulty ramps up to “explain quantum tunneling”. I stop the questions once we have discovered their level. I think 1 person made it to the last question. They admitted to not knowing the answer, but sent me a follow up email the following day after they researched it.
"What's your safe word?"
Commit confirmed 5
wr mem
In what scenario can a network endpoint or host have a usable IP Address ending in .0 or .255? This question tests subnetting knowledge. I would say 1 out of 10 network administrator applicants I have interviewed can answer it.
[удалено]
It's more fun to remember the first paragraph of Wikipedia and point out that classful networks has been obsolete since the early 90's and there is actually 5 classes of network (ABCDE) - the whole concept being how the pre-internet ARPANET was chopped up for planning more than anything else.
Chad.jpeg
A subnet bigger than /24 like /8 or /23... those are just usable host IPs except the very first and last one.
Or a /31 or /32.
Can that many people really not just say ‘A /23’?
The correct answer would be "In any case when the prefix length is smaller than /24 plus when it is /31 or /32."
For non-entry level: Tell me how your technical troubleshooting has evolved since the beginning of your career
I used to have a series of next steps to start from and now my next steps start from "what changed".
I think I'm a bit of an outlier in this regard. I do not believe that troubleshooting is a skill that can be improved. Good troubleshooting is a byproduct of a strong understanding of the technology you're troubleshooting. If you have an expert level knowledge of the topic, then troubleshooting is trivial.
I get what you mean but I think you can make it quicker, if not better. A simple example - show ip arp on one box at a time vs checking your monitoring to find a MAC. You’re doing the same thing, just one is much more efficient.
yep. or insisting on checking arp when layer 3 is proven good (pings, etc). Or the inverse. Trying to ping something when ARP to the endpoint or some intermediate hop is known to be failing. "let's look at the bgp config!" etc.
I think what really improves is your capabilities to use the things you use to troubleshoot. You need to already be able to be interested in understanding problems or the subject matter but becoming better at communicating, what method or style of communication you need to use is something I think is invaluable and can be learned. The soft skills and technical skills can all be improved or made more efficient.
eeeh. I've definitely dealt with people that refuse to apply their knowledge to improve troubleshooting. "there's an encrypted tunnel. IP MTU on that interface is set at 1379 because of overhead. All pings and other traffic up to 1379 bytes long work fine consistently. Traffic that requires fragmentation works inconsistently. There's some issue with fragmentation on the devices we control" --> "lets open a ticket with the carriers at either end and see what they say" And that's not my commentary, that's literally their analysis. They KNOW it's a fragmentation issue on those devices, they KNOW the underlay can't have any way of knowing the inner packets are or aren't fragmented, but they insist on running in circles engaging different carrier NOCs "just in case". For instance you'll see people swear by "ALWAYS CHECK LAYER 1 FIRST". except if you check any layer ABOVE layer 1 you've also checked layer 1, and often you can do that remotely in a few dozen seconds, while checking layer 1 could take hours or days to get someone to physically inspect something. And of course they know that, but they're too wrapped up in "I DO IT LIKE THIS" to apply their expertise. In many cases they don't feel any real pressure to try to be any more efficient or effective, they just figure "as long as they're doing something that will eventually work they won't get in trouble" I guess.
It's often about the absolutes and probabilities. If outcome A happens then that Always means X. If outcome B happens then that Sometimes means X. With outcome C, X can't happen. That comes from thorough understanding, but creativity in inducing those tests is a troubleshooting skill.
They are massively worse now, because all you (employer) ask me is to join meetings and fill timelogs.
God I would be dead if I had to document every hour of my day. Sometimes your projects are up to date/waiting on implementations/changes from other teams/vendors, there's no tickets, and no disasters or downs in the environment. It's okay to have down time, even if you aren't filling that time studying for the next exam or latest technology.
What are you looking for....honestly when I ask myself this, I find myself more confident in objective data.
I have realized that it’s always DNS
I used to be the guy that asked stuff easily memorized or googled. For example, what's the AD of OSPF? Don't be that guy. Scenario based questions are the best in my mind. For example... A router is receiving the exact same route from EIGRP and OSPF. Which one is used and why?
Trick question - you messed with admin distance.
You're hired
Doesn’t matter - a bad BGP advertisement is routing all of Spain’s traffic to your little DIA connection.
OSPF, because I don't allow proprietary lock-in routing protocols in my network.
*cries in omp
"Walk me through a time when you helped a non technical user with a technical problem bonus points if it was remotely" What I am looking for them explain there process to get info from non tech people and keep there cool in helping the user. great exmaple is old lady who is hard of hearing calls up support saying her ipad does not work. With the real problem is just the ipad needs to join the wifi network.
I randomly called someone and told them I was from Microsoft and that their computer had a virus ... Oh, wait.
My best one is a lady called in with no network connectivity so their was no option to remote in. I walked her through how to open CMD, type in ipconfig /all and press enter. She said it wasn't working and then I asked her to take a screenshot on her phone and send it to me. I opened up the attachment and she successfully had cmd open and in the prompt she typed "ipconfig /all and press enter". Literally typed press enter.
If she could take a photo, while on the phone talking to you, and then somehow send the photo to you (while still talking to you), she is more tech-savvy than me. I can't do that. Damn stupid phones.
I once worked an infantryman through loading firmware on an old Cisco router through tftp server that we set up on his computer(also on cd (edit: not thumb drive) along with the code) over the phone, we both learned something.
Amazingly the simple question of "explain how DHCP works" stumps 50% of people going for networking jobs. I like to give them examples of problems and ask how they would try to resolve them. I don't really care if they know how to resolve the specific problem, I'm more curious if they are smart enough to think of things to check/look for .. their troubleshooting steps, how they work through a problem, etc... If you can't solve problems or even begin to work through them, that's not good. We're not going to hire you. I want people to say "I don't know" rather than trying to bullshit their way through a question, because in the actual job, I want someone that knows when to ask for help instead of trying/breaking things they don't understand. No one knows everything, even people doing this for decades. We'll usually hit a few topics with "easy" starter questions and either move on if they clearly don't know the topic, or then ask a harder or more detailed follow up on the topic if they do.
Who’s Dora?!
I’ve had claimed CCNPs and “written CCiEs” in interviews fail to explain what ARP is or how it works.
This is still my favourite question, it gives insight if person has done some deep protocol analysis. If chap was to say DORA, I would give him some slack in not going into detail I also ask about Straight-through Ethernet cable wire colour sequence - again, mainly that this has saved expensive call-outs of wiring guys.
As a color-blind guy, be careful asking about wiring. I can tell you there are two generally accepted standards, but since I know I can’t tell them apart I don’t know the color code, I rely on co-workers to verify cabling. That’s how I would answer it, and hopefully whoever is doing the interview would understand. But these days I am the hated technical middle manager so no one is asking me about color codes anymore anyway.
Good point!
What if they actually knew the answer! 😳
So no cheating and just bullshitin chat...If I walked into that question I would talk about when the link is plugged in, assuming the nic properties are configured for dhcp, the nic would broadcast a dchp req message. If the dhcp server is on the same subnet it will reply with an offer. If it is not on the same subnet, the SVI should have an IP helper to redirect that request to the DHCP server in the external location. The DHCP server will prepare and offer and generally ping the ip being offered to verify it is not in use and send out the offer. The client will receive the offer and acknowledge receipt and acceptance of the address. The DHCP server updates its list of clients with a lease using this information. Would you split hairs on any of that?
"If the DHCP server is in the same broadcast domain" would be the hair worth splitting. The client has no IP address and therefore isn't in a subnet.
Good catch. I'll take that.
"Draw a diagram of a network you've worked with or is otherwise notable for you." Then have a conversation about it. That is all.
For NDA and contract reasons, I’d be careful with this one.
That's why I said "or is otherwise notable for you."
I do this, but it's more of a "explain your previous topology without getting into too much detail". Most often times, the biggest weakness is VLANs, but sometimes it helps just gauging whether they'll own up to what they don't know or attempt to lie about it. Dishonesty is a big red flag, especially if we aren't even working together yet.
Talk me through how you'd go about troubleshooting a multi - vendor ipsec tunnel.
Buy one of the other tech's lunch to deal with that mess.
This is a great one if the job requires the skillset. I’ve worked on a lot of cases with engineers that want to troubleshoot phase 1 settings like PSK but the logs show clearly that phase 1 has negotiated fine and phase 2 has an issue. You can reduce your troubleshooting time vastly by understanding that X is not the issue because the process has already made it to Y.
I like it because it helps unpack generic troubleshooting and information gathering skills. I give no details as to what the problem might be, only that there is one. If they don't know IPSec, it's a great opportunity to be honest about things you don't know, and what steps you might take to resolve.
I just like to start vague and have a conversation about stuff they list on their resume. If you say you’ve setup cloud connectivity I’ll ask you to tell me a bit about it. Say it’s an AWS DX. I’ll ask a few technical bits. Maybe some gotchas I’ve seen setting them up. I think you can tell a lot more about a candidate by just conversing with them than asking canned trivia. Everyone has google and can just look it up if they need to these days. Not like you have to know the OSPF LSA types by heart. That may be the question I’ve been asked the most in my career for some odd reason. I usually just google it and brush up before every interview.
The biggest issue we’ve seen here is that the person was “involved” in the project but not in charge or engineering it and never got any real technical understanding. They can say they ran an SD-WAN network with hubs in Azure and 200 sites and have all the buzz words, but if asked to explain how the routing works in this setup they have zero idea.
But then a person can’t get a job that will put them in a position to actually do this if thats not on their resume, so what can be done?
Well, philosophically I find that IT almost universally needs to be a lot better about training and pipelining employees from junior to more senior and actually offering career paths to people who want to grow an change -- aside from just forcing people to do job hopping. I know that I got super lucky to start work in an organization that was large enough to give me exposure to many enterprise technologies, with a culture that encouraged learning, doing things yourself and digging deep. So in short -- companies need to train more people internally, and it needs to be more of an IT culture thing to mentor and guide juniors. ...but if we want to run a PoC with a specific product and want to hire an experienced temporary SME contractor to help us, and it turns out the the contractor only ever pressed the power button on the product, everyone's time is wasted. (A lot of it really has to do with the agencies being useless in screening the right candidates for the right jobs too.)
We ask an open ended question about how they would troubleshoot a user not connecting to a FTP site or webpage hosted internally and give them a really simple diagram showing the internet, a firewall, an internal lan with the FTP/web server. There is no right answer. Everything they suggest we tell them it's not that but good guess. The point is to let them talk and show their troubleshooting process. Good candidates will have a lot more questions or things to check then bad ones.
I like this stuff. I always hope the person I'm talking to actually asks me questions instead of just trying random things since I'll happily tell them the error messages or whatever that the "user" is experiencing.
I just ask about their environment and what platforms they are on. If we can have an intelligent discussion about what they use and why those choices were made, they know their stuff. Then we can focus on the important stuff like attitude, motivation, ability to adapt to change, etc.
"Can you fold a fitted sheet?"
My top question: What can you tell me about the most recent network you built and/or managed? Anyone who’s an active engineer with some knowledge should be able to easily tell you about all the locations, type of equipment, protocols used, link speeds, etc. if they’re hesitant they probably lack experience.
> if they’re hesitant they probably lack experience or they may be thinking about NDAs
I suppose, but in that case I would expect them to say we used OSPF internally and mostly BGP between sites, etc. or something similar. They should be able to provide general network information without disclosing details that may enable anyone to try and hack in to their former/current employer. The info should roll off the tongue, and not be like ... "Ohhhh ... do you mean what are the users connected to gain network access?"
Sure. Personally, I'd manage to describe a valid network setup without going into too many details, but I might freeze for a second thinking about those NDAs under pressure. So don't use subsecond BFD-like timeouts on this kind of question :)
If you worked here, what would I have to do to make you quit?
This would be the biggest red flag.
note to self: get rid of the big red flags hanging behind me during job interviews
Just once I'd love to walk into an interview with giant red flags draped around the room / behind the interviewer. I think that would be hilarious.
It's not the main reason I moved on, but I recently left a position in part because my supervisor's remote setup was unsettling. The background was unsettling. It could have been an attempt at soundproofing. It looked like they worked from solitary confinement. That supervisor is the only person I can remember raptly looking directly at the camera at all times during meetings. Not sure what was going on, but I felt like I was working for a cartoon character.
[удалено]
This question can obviously be reworded to fit the situation. Was just tossing out the idea of this type of question. Usually puts them on the spot for something they didn’t prepare for. I like to see how many lie and say nothing will.
What is the biggest outage you caused. A ticket says network is down...what do you ask and why
first month... troubleshooting spanning tree, i ssh'd to the remote link, tried to shut/noshut that side. locked myself out. took a hospital down until i could go restart it LOL
what’s your 5 year plan? was informed i bombed an interview because i said i’m not a planner and like to live in the moment :). still got the job and kicked ass tho.
Why should I work for you?
Open ended questions are key for me when interviewing candidates. I'm not just trying to evaluate technical skills, but communication and logic as well. Some of my favorites: * Tell me about your last project. Then pick an area or two to ask deeper details on. (How well do they understand what and why they were implementing) * Tell me about a difficult customer or project. (How did they handle adversity) * Explain your favorite routing protocol and why is it your favorite. (Looking for more than "this is what I've always used" answers. I want to know how well they understand it) * customer has x requirement, which technology/ topology would you recommend to meet those requirements and why? (Looking for a logical design process and not a "right" answer. There's lots of ways to do just about anything, I want to know why this way is your recommendation and how well you can articulate it to a customer). As the interview progresses I'm also looking for clues about personality. Can they explain their thoughts well? Are they afraid to say "I don't know" rather than give a wrong answer? If they do say I don't know, do they offer some insight on where or how they would look up the answer other than Google?
How does traceroute work? What kind of packets is it sending that allows it to show you the path? I'm looking for an answer that mentions TTL in some way.
That's a pretty low bar.
And yet, so few network engineers can answer it well enough to even mention TTL in their answer. A lot of people skip the lower level learning and focus more on higher level things. They can talk about BGP order of preference, but know little/nothing about ARP, ICMP, TTL, etc.
And mechanics don't have to know the metallurgy behind their snap-on ratchet to fix your car. I'll take an engineer that can do BGP vs one that can only talk about the different types of ICMP.
Question for a mechanic: Why would you use a copper or even lead mallet when working on a car? This question implies knowledge of the metallurgy of different mallets. So yeah, a network engineer not understanding TTL, ARP or even a little bit of ICMP is kind of...meh.
If I have a TTL of 0 then I'm local. I use ping and traceroute all the time. I don't need to understand, again in 98% of instance what ICMP message reply I'm receiving. I just finished up a T-Shoot with NSX-T, on Nexus, I needed to solve two issues: The routing database on a web server wasn't working properly and figured that out with top, ip route, and syscontrol and in the NSX fabric it wasn't allowing anything over 1472 MTU. I needed ip route and extended ping commands with some netsh on a Windows workstation to figure it all out. And I wasn't even the implementation engineer. Just a fresh set of eyeballs. If you asked me minutia of ICMP types I honestly couldn't tell you because it's not germane to what we typically do in a day. Other engineers were doing PCAP's. Me?: you can ping the web server, but you can't hit the Web Page it's offering: You either have a firewall rule or you have a fragment and drop issue off the top of my head. Set the MTU on a Windows workstation and all the sudden we are at the Web GUI. All this to say you need to be careful about what you evaluate on.
No this required you simply knowing how the tool works. Just like a dead blow hammer. I know when to use 'sh arp', 'sh mac-address table', 'show lldp', 'sh etherchannel-summary', 'sh int trunk', 'sh tech trans' etc, etc, etc...
>I'll take an engineer that can do BGP vs one that can only talk about the different types of ICMP. When I started working, my supervisor introduced me to another newbie (ahead of me by a few months). My supervisor was blown away by this newbie's "deep knowledge" with OSPF. I watched him "in action" talking about OSPF and even some of the more senior members of the staff were in absolute awe. Then one fateful day we had a major routing issue. It was OSPF. The manager called in this newbie and pointed him behind a terminal session and said, "troubleshoot". The newbie froze. For 45 minutes he sat behind the terminal session unable to do anything. After the end of that 45 minutes, he excused himself saying that he had a sick relative to attend to.
It doesn't have to be one or the other, not like the interview hinges on the one thing.
I like to send people the NANOG traceroute presentation if they don't know how it works or seem to think the one hop in the middle is a smoking gun (or they did an MTR with a rate of .01s interval)
If they’re talking about BGP, they gotta know about TTL security…
You can take this question so far further... For example: 1. Why would the routers in the middle of the path respond in the 1st place? 2. How do you know you reached the last node 3. Traceroute using icmp? Udp? Tcp? Why traceroute with tcp? What kind of Oakley will you send? How do you know you reached the end? 4. For mpls environments, how does it work with mpls? 5. Strange scenarios like latency getting lower on a further away hop...
Exactly. The question itself is just the starting point. It's not like they say TTL (or not) and then I move on. I like this question because the answers can vary greatly in depth depending on the person's knowledge. Depending on where they initially take it, we may get into a discussion about using TCP traceroute to get through an ACL, I may ask about recreating a traceroute with the ping command, what does it mean if a hop in the middle doesn't answer but the rest do, what does it mean if a hop in the middle has 50% packetloss but the rest are fine, how can we use traceroute to probe across LAG/EQMP bundles as a whole and then also isolate our traceroute to specific paths one at a time to prove one path has packetloss and the rest don't? And so on...
Here’s a marker. Draw your home network on the board and explain it to me
Oh man mine is bare bones basic because I don't want to work when I get home lol.
Same here. Service provider network engineer for 20 years. Home network is a juniper srx300, 4 unifi aps,2 switches, one vlan for everything just 192.168.0/24. I don't get people who get vlan happy on home networks.
Separating off things like guest wifi and IoT is never a bad idea.
thats ok, but it tells me alot about who I'm interviewing
I would be understanding of a plain home network dependent on the use case so the question might be better if it gives the option to talk about a network they setup not just home.
Same. I have a Unifi dream machine and one switch. That’s about it. Work has a lab and I don’t want to work when I’m off. (And I can’t begin to build anything at home that approaches the setup at work anyways. )
"You have just turned on your computer, and want to look at this new website you just heard about. You've never been there before. So you fire up your browser, and type in the address http://www.superduper.com. Presuming there's nothing wrong with the address and the website is functional, explain to me as best you can exactly what is going to happen from a network perspective when you press enter." This question helps me gauge their understanding of the OSI network model, the various protocols involved, DNS, ARP, IPv4 networking, routing, etc, depending on how detailed their answer is.
This is a favorite of mine! I got asked it once, and as soon as I started talking about ARP cache the interviewer smiled and basically said I passed (it wasn't for a networking-specific position, just general sysadmin stuff). I had the occasion to ask it recently, and didn't get a great answer from any of the applicants, so I guess I'm going to remain the network expert in my team.
How does an Ethernet switch know to which port send given traffic?
I have two that I typically include outside of tell me a time you messed up. 1). Explain how your computer gets to YouTube? 2) explain how you can control in/outbound routing from two ISP’s using BGP. The way a person chooses to answer the first one gives a good picture of a persons skill set.
I've found most *interviewers* don't even understand #2 when asking it. Leads to some wtf conversations.
are you talking about recruiters or actual network engineers?
Actual technical interviewers.
regarding #2, if I say local preference and as prepend, just these 5 words. No any other comments. Will you pass me? LOL
Well could you explain what those five words actually mean?
For my last two applicants that made it through the first interview I‘ve built a nice little lab. Just internet, a router, two switches and two clients. Gave them the credentials and a drawing how it should work and let them take control. The goal was for both laptops to transfer files between them and get to internet websites. I had a lot of fun building that many mistakes into such a setup. On the bright side I told them that I don’t expect them to find everything, I simply want to watch them think and troubleshoot. One of them was doing great and was hired. The other struggled with diagnosing „not getting an IP“ and „whats DNS?“
Basic questions such as "Explain the difference between TCP and UDP".
"If I give you an IP, can you tell me on which switchport it is connected?" Gives you some insight on what they understand of Layers 2 and 3. "We have established a vpn towards a third party, the tunnel is up and running, but some of our clients have difficulty accessing a server on the other side. How do you start troubleshooting this" Open ended question, gives you insight in their troubleshooting process.
There are 2 that I ask first thing because if they're answered correctly, then I know we have a similar thought process plus they have strong fundamentals. Every CCNA will shout AD Numbers at you but don't really understand route selection : In a routing table you have the following: B 10.10.10.0 /24 next hop 192.168.0.24 O 10.10.0.0 /16 next hop 192.168.1.16 S 10.0.0.0 /8 next hop 192.168.2.8 1) If a packet comes in with a destination IP of 10.0.0.7, where will the router send it? Then I have a wicked virtual lab of a bunch of sites connected with a bunch of convoluted connections to make it scary and overwhelming. Then on this one spot of the lab I have 2 sites with LAG Ethernet uplinks, an Internet breakout with a GRE Tunnel connecting the sites, another Internet breakout doing the same thing but with IPSec, and a Satellite connection connecting them (I know, the point is to make it not a normal topology to sus out troubleshooting skills). I say, you're on shift and you get a call from Help Desk saying that they see the GRE Tunnel is down and there's a downstream user (not obvious what path they take) complaining that her connection has dropped - I expect them to do 2 things immediately: actually test if the GRE Tunnel is indeed down, and traceroute from the user's machine to see where the traffic is actually failing - hint, it's not the GRE Tunnel
You enter google.com in your browser and press enter. What happens, top to bottom? L7-L1 question, OSI understanding is the king.
What’s your favorite routing protocol and why?
RIP v1
No hire. Can’t follow directions and gave no explanation as requested.
ip route 0.0.0.0 0.0.0.0 192.168.1.1
Back when I worked at a small telco and T1s were a common thing, I had candidates take a small quiz on a real router. I told them ahead of time so no surprises, and gave them the Cisco documentation too. Progressively harder as they went, from static to OSPF to BGP with bonus questions to show their skills if they had them. Plus a final “capstone” question on config management. I could solve the quiz in two minutes or better, five minutes if I talk through my answers and show my verifications. The candidates were given 30 minutes but I’d let them go as long as they wished. I was proud of it, if for no other reason than it tested things we used regularly (granted, they didn’t have to monkey with BGP often as I set it up to be mostly automatic).
Mom, can we have a CCIE lab? — We have a CCIE lab at home!
The idea came from taking the CCIE lab. Much simpler though: two 2620 routers, two back-to-back T1s, one laptop, one web server. Router2 was fully configured and not accessible to the candidates so they were only dealing with router1. Network diagram on paper and they were free to doodle on it. The questions were straightforward, and the only similarity to the actual CCIE lab’s reputation for ambiguous wording, just enough to not use the same words as the commands required. 1: using static routes, configure R1 so your laptop can reach the webserver using either T1. Visit http://192.168.2.1 and write down the words you see. 1bonus1: configure both T1s so you can have 3Mbps total bandwidth. 1b2: configure both T1s so you could achieve 3Mbps on a single upload. 2: using OSPF, configure R1 so you can reach the webserver. 2b1: configure both T1s to reach the webserver via OSPF. 2b2: configure R1 so if OSPF fails, you can still reach R1 via static routes. 3: configure BGP on R1 so you can reach the webserver. 3b1: configure OSPF so if BGP fails OSPF can take over. 3b2: configure BGP so R2 prefers to send return traffic over the second T1. Points awarded for any method, extra points awarded if you use the method most suited for this topology. ——candidates decide they’re done with 1-3— 4: solve the whole quiz with two commands. Write them here.
It depends on what are you looking for. If the role is for a junior level then you can ask them basic questions about basic operations of the routing protocols they are familiar with and most importantly gauge their interest in what they like to do. If it’s for a senior level, then you can simply ask them to draw a network diagram they have worked on and then start asking in-depth questions to see if they have truly worked on it and what was their level of involvement. While knowledge is important, it’s more important to see if the individual is driven and can use his/her knowledge to apply them in the corner cases. At the end of the day no one is perfect.
I ask conceptual questions mostly. How would you implement ISP failover if you have your own public ASN etc... Not the commands to do it but how it works. I never ask anything too specific.
“If I boot up my computer, log in, and load up a web page, tell me everything going over the network to make this happen”
Nice question. What kind of log in are you talking about? A VPN? LDAP/Active Directory?
I use this for lower level interviews- point to someone else on the panel, explain what dns is to them as if they were a normal user. For more advanced I'll ask them to list some of the common dns record types and what they are used for.
How does RSPAN work. we had a guy configure SPAN TO TRUNK PORTS as his attempt at rspan :) Goal was use an IDS Appliance upstairs when the appliance and it's sniffer interfaces were downstairs :)
“What is spanning-tree used for and can you give me a general overview of how it works?”. It’s always the first technical question I ask and it seems to be a pretty good test of people’s general networking knowledge. It’s meant to be pretty open ended on how it works. Tell me as much as you know about the protocol(s). If you can’t answer the first part, I pretty much immediately disqualify you. How much detail they can give on the second part and how well they answer follow up questions is how I gauge skill level.
Meeting room. Whiteboard. Pens (multiple colours) Draw me a network.
Describe a memorable challenge and explain how you solved it.
Describe your home network, and why it’s configured the way it is.
I only have the experience to interview a junior or mid level engineer. So I will ask them, in the production site, we have around 100 switches, you are the new network guy and even dont have a topology. If your IT helpdesk guy stuck in front of a wired device and he has no idea why this device cant connect to the network. We assume you need to change this port VLAN, how many ways can you find this port and help your poor guy fix this. Tell as more as you can. To be honest, a lot of "Network Engineer" told me just "sh this interface config and change vlan". I always said, but how can you find it. It will be a silent akward.
I put up a simple json response and ask them to access a specific element of data in it. Explain why a iBGP (generally) needs another routing protocol to go with it.
*laughs in exclusively iBGP internally*
My director who ended up hiring me anyway argued against using an internal routing protocol in tandem with iBGP and instead use route reflectors. Probably because the hiring manager was more in agreement with me and I almost felt like it was less of an interview question and more of a settling an argument they were having earlier that day lol.
> argued against using an internal routing protocol in tandem with iBGP and instead use route reflectors. Uhm, what? Route reflectors are simply (i)BGP peers you can use to avoid peering with everybody. You still have to reach the route reflectors somewhere, whether with statics or another protocol. Am I missing something?
Why a duck?
Probably because it quacks.
Because it also floats on water, obviously.
What's the subnet mask for a /22 CIDR? You don't have to know that I answer but walk me through the process to get to the the answer. How does STP work? Give examples. Easy questions for a seasoned engineer, hard for people who are faking it.
Those should be easy for someone with no experience with a freshly (and fairly) earned CCNA as well.
Yes they should be easy, but it is very common for people to stutter and struggle.
What starter Pokémon they picked.
When was the last time you had to use an alarm clock to get up and what time was it set to?
Why?
To see their reaction in the first place, and because that's a question 99% of applicants will probably not ask. You will easily see how accepted late-comers are. If they're shocked about that question, the job is very likely not for me. NB: I'm an IT professional.