Lead Network Operations Engineer

About CENIC: Join the team that connects California to the world — advancing education and research statewide by providing the world-class computing network essential for innovation, collaboration, and economic growth. Our organization operates the California Research and Education Network (CalREN), a high-capacity computer network with more than 8,000 miles of optical fiber. The network serves over 20 million users across California, including the vast majority of K-20 students, together with educators, researchers, and individuals at other vital public-serving institutions. Bring your IT career and technical expertise to help support the California Research and Education Network (CalREN), a high-bandwidth, fiber-optic network that serves over 20 million users. CENIC is a nonprofit governed by its member K-12 schools, libraries, community colleges, universities, and other public-serving institutions. Its mission is to advance education and research by providing a world-class network essential for innovation, collaboration, and economic growth. We are looking for Network Engineers at various levels: mid-career, senior, technical expert, and leadership roles! Please see our career center for other positions available. Position Summary The Lead Network Operations Engineer is a senior technical individual contributor and shift leader within CENIC's Network Operations Center (NOC). This role combines deep, hands-on expertise in large-scale IP and optical network operations with direct responsibility for the performance, accountability, and output of the network operations team during an assigned shift. The Lead Network Operations Engineer guides and supports a team of Network Operations Engineers and Administrators, serving as the primary escalation point and ensuring that all shift activity, from routine operations to major incidents, meets CENIC's standards for quality, timeliness, and documentation. This role requires mastery of CENIC's network technologies and the operational judgment to make real-time decisions that protect the integrity of CalREN and the services it provides to over 20 million users across California's research and education community. Supervisory Responsibility Direct supervision and mentorship of Network Operations Engineers and Network Operations Administrators assigned to the shift, including task delegation and workload management Own shift-level accountability for team performance, incident response quality, and adherence to NOC processes and procedures Conduct weekly reviews of all ticket updates, providing structured feedback and ensuring documentation standards are met Ensure all supervised personnel are aware of and operating in accordance with current NOC processes and procedures, both established and evolving Lead the documentation of shift turnover, ensuring an accurate, complete, and actionable handoff to the incoming shift - including open incidents, ongoing work, and any items requiring follow-up Serve as the primary escalation point on shift and escalate to senior management or engineering as required Conduct regular one-on-one meetings with shift staff to support professional development, address performance, and identify continuous improvement opportunities Track and report on shift- and NOC-level metrics, KPIs, and operational trends to support continuous improvement to improve network reliability, workflow efficiency and transparency, and engineer accountability Cross-departmental collaboration across CENIC to drive business needs and network improvements to better serve our member institutions Essential Functions Serve as the senior technical authority on shift, leading the team's response to complex network issues, major incidents, and service-affecting events from detection through resolution Apply expert-level knowledge of DWDM, IP routing (BGP, OSPF, IS-IS), MPLS (Segment Routing, or LDP), and VPN services (L2VPN, IPVPN, EVPN) to diagnose and resolve incidents on CENIC's large-scale networks and optical infrastructure Troubleshoot and resolve issues across CENIC's DWDM optical line systems, DCI deployments, and transport layer, including physical layer faults and capacity events Lead incident management during major outages: coordinate team actions, manage member communications, escalate to senior leadership as appropriate, document the incident timeline, and ensure a thorough post-incident review Maintain shift accountability for all open tickets and work items: review ticket quality, ensure timely updates, and verify that issues are resolved or properly handed off before shift end Oversee the execution and documentation of a complete shift turnover at the end of each shift, briefing the incoming shift on all open incidents, pending escalations, scheduled maintenance, and anything requiring monitoring Proactively monitor network health; identify emerging issues, traffic anomalies, or capacity concerns before they become service-affecting events Contribute to the ongoing development of NOC processes and procedures, identifying gaps, proposing improvements, and piloting changes Identify opportunities to reduce manual, repetitive operational tasks to improve operational workflows Serve as a technical resource and mentor for engineers on the shift, supporting knowledge transfer, skills development, and consistent application of operational best practices Keep current with emerging trends and technologies in research and education networking; apply relevant developments to improve network operations and CENIC's network capabilities Lead the planning and execution of maintenance windows, network upgrades, and equipment installations, including occasional travel to CENIC facilities or member sites Track and report on vendor performance to gather detailed RFOs, resolve chronic circuit issues, and provide feedback for annual business reviews Key Success Factors Expert-level technical knowledge of large-scale IP and optical network operations Strong shift leadership and team accountability: sets clear expectations and holds the shift accountable Sound operational judgment under pressure, with the ability to make decisive, well-reasoned decisions Disciplined and timely documentation habits: produces clear, accurate ticket updates, turnover notes, and operational records Excellent oral and written communication skills, including the ability to communicate clearly with members and leadership Organized and capable of managing multiple concurrent issues and priorities Strong commitment to team development and a culture of continuous improvement within the NOC Collaborative mindset when engaging cross-functional teams and external partners Education & Experience 5+ years of experience in network operations and engineering at a CCNP/JNCIP level or equivalent; 10+ years at a CCIE/JNCIE level preferred Deep working knowledge of BGP, OSPF or IS-IS, MPLS (Segment Routing or LDP), L2VPN and IPVPN services (EVPN or similar), and traffic engineering in a large-scale production environment Hands-on experience with DWDM optical line systems and DCI deployments Demonstrated experience operating large-scale IP and optical networks in a 24x7 production environment Strong vendor-agnostic knowledge of current networking hardware platforms, including experience with equipment installation, upgrade, and hardening Prior leadership experience in a NOC or network operations environment preferred Four-year degree in computer science, information technology, or a related field, or equivalent work experience preferred Experience in a research and education network environment or a comparable large-scale, mission-critical network is preferred WORK ENVIRONMENT: This position operates in a professional office environment with standard office equipment, including computers, telephones, photocopiers, and filing cabinets. WORK HOURS: Network Operations is a 24-hour, 7-day per week, 365-day per year operation. An employee's regular work schedule is four ten-hour days on any of the seven days in the week, based on CENIC's business needs. WORK AUTHORIZATION: Documented Proof of identity and legal eligibility to work in the United States. Applicants must be authorized to work for any employer in the U.S. We are unable to sponsor or take over sponsorship of an employment Visa at this time. CENIC does not offer STEM OPT training opportunities and cannot complete Form I-983. WHY WORK AT CENIC? Company Culture: Small teams and a collaborative environment provide the opportunity to make a significant impact in a small organization. Community Engagement: We work with research and education institutions in the largest state economy in the United States. Your work has meaning. Generous Health Benefit Package: We value our employees' wellness. We offer a variety of medical, dental, and vision benefits. We have you covered with life, short-term, and long-term insurance. We also offer pet insurance at a discount. Paid Time Off: We offer liberal vacation and sick leave, plus 15 paid holidays. Investment in Your Future: CENIC contributes to your 403(b) even if you don't. We also offer matching contributions. Pre-tax and Roth options are also available. CENIC is an equal opportunity employer – Vets/disabled