Monday, December 26, 2016

Top ten from 2016, part 2

I'm doing a year in review, and re-sharing some of my favorite articles from 2016. I posted the first part of the list last week. Here is the second set.

6. IT organizations must adapt or die
Think about how much things have changed since that application first went live in 1998. Back then, most of us used desktops. Laptops were available, but in the company where I worked, only the CEO and CIO used laptops. They were too expensive for the rest of us. Cell phones were common, but they were big, blocky affairs that only made phone calls. Technology is always changing. You don't have to go back very far to see how quickly technology evolves. Ask yourself how things will be different a few years from now.
7. Career ladders
A few years ago, I served on a committee to identify new career ladders for our IT teams. This work originated with a document I wrote about how to "level set" the compensation and seniority for our operations and infrastructure staff, many of who had joined our team as part of IT consolidations from different parts of the organization. In that document, I set a new career progression, from "junior" to "staff" to "senior" to "team lead," with compensation ranges for each. This document was going to be the basis for a re-organization of my teams. I have since used this career ladder concept as a basis for organizational planning. The document provides a simple overview of technology teams and how job areas are aligned and related. Our work was expansive. We defined eight separate career tracks, with different levels within each.
8. How to lead an affinity exercise
My job is to provide leadership to my IT organization, but often that leadership involves soliciting ideas from a larger group. You might lead a governance group, or you might meet with your entire team as part of an annual meeting. It's good to leverage meeting opportunities to caucus the group and tease out common ideas. But how you solicit ideas may differ depending on the size of the group. For small groups, I often use a SWOT exercise. In larger groups, I rely on some variation of an affinity exercise.
9. Good and bad bosses
We've all had good and bad bosses. Hopefully, you've had more good bosses than bad bosses. Nothing can sap the energy from you like a boss you don't like. On the bright side, a good boss makes you feel valued and excited to come to work every week. No matter what bosses you've had, I hope you learn from them. What best practices do you observe that you can adopt for your own management style? What bad habits do you notice that you should avoid? In this light, I'd like to reflect on several managers I've had over my career, both good and bad.
10. Yes but no
A former colleague from the University of Wisconsin-Madison once showed me how to build trust within teams through the "Yes and" approach. When you say "Yes," you provide agreement. If you build on that and say "Yes and," you lend support.I prefer "Yes and" statements. They are positive, reinforcing conversation tools that build trust. In contrast, "Yes but" statements erode trust. Some people seem to think it's the same as "Yes and," but it's not. "Yes but" is at best a way to stay on both sides of an issue. At worst, "Yes but" is a negating statement. The "Yes but" statement says "Yes, I agree with you, but not really."
That's it for 2016. Happy New Year!

Monday, December 19, 2016

Top ten from 2016, part 1

2016 has been a great year. I was fortunate to take on a new position that I love, and I moved back to the Twin Cities. As the year winds down, I like to reflect on some of my favorite articles. Here are my top ten, in no particular order:

1. Advice for new managers
When I started work all those years ago, a colleague gave me a little book filled with aphorisms about life in the office. At first I thought it was a humor book, with comments like "When the office secretaries say they are cleaning out the fridge of anything older than one month, it is time to grab your bottle of salad dressing and put it on your desk." But in the first month of my first job, I quickly realized the truth behind these pithy observations. When I later became a manager, I wished someone had shared similar wisdom with me about how to act as a new manager. So I would like to share a few brief observations that may help first-time managers.
2. Why executives use Powerpoint
In my new CIO role, I'm in a much larger organization. I'm always meeting with people. That in-person contact is very important to me; I like to build relationships with those I work with, as a way to get things done. When I'm not in a one-on-one meeting, I'm usually in a committee meeting or a steering committee meeting or a governance meeting. These are important meetings too; they are the mechanics of projects. But the side-effect of all those meetings is that I rarely have time to write documents. So instead of writing documents, I prepare a Powerpoint slide deck, and I speak to those issues when I'm making my presentation.
3. Five levels of performance
At what level is your organization performing? Are you exceeding expectations? Falling below expectations? Or just meeting the expectations? Some experts refer to four levels of performance, originally credited as the "Four stages of competence." Others refer to five levels, with the fifth level indicating either an ability to instill performance in others, or a "super-performance" level. I prefer the latter. These five levels of performance apply to both individuals and organizations.
4. Understanding risk
Throughout my career, I have tried to take a risk-based approach to actions and decision-making. This has become more important to me as a CIO. Understanding risk is an important part of driving change. If you understand the risks, you can decide which risks you can accept and which you cannot. In doing so, you avoid "analysis paralysis" where you continually evaluate options without actually choosing a direction. By taking a risk-based approach, some options become obvious.
5. About governance
No matter how IT is organized, you must always consider how IT is governed. How do you ensure that IT is meeting the needs of the business? This governance can be formal or informal depending on the relative maturity of the business and of IT. But at some point, IT needs help to "vet" IT decisions to best serve the needs of the business. Governance can take on many forms, but the general process is that someone listens to business needs, and a governance group prioritizes requests and creates projects to execute them.
I'll post the remainder next week.

Friday, December 16, 2016

Leadership lessons from unusual places: Project Runway

With this post, I'm wrapping up a week-long recap of my presentation "Leadership Lessons from Unusual Places," from last week's Government IT Symposium.

Coaching and mentoring are very important to me. I named my blog "Coaching Buttons" because of the importance of coaching. The name "Coaching Buttons" refers to those informal moments when you can provide some coaching to one of your team.

Of course, it's better to find separate time for proper coaching. And there's a great example of coaching in Tim Gunn from the TV reality show Project Runway. If you haven't watched the show, it's a reality show where fashion designers compete. Because it's a reality show, there's an elimination every week. But I think it's fair that designers get eliminated based on merit, not personal bias from the judges.

Each episode has the same basic structure: the designers receive their challenge, they shop for materials, then they have one or two days to create a fashion garment. It's a very short timeframe for the designers to create and execute their visions.

About half-way into each episode, Tim Gunn arrives to provide some coaching to the designers. Let's watch this brief clip and observe Tim's coaching style:



I think this clip is typical of Tim's coaching on the show. Note how Tim asks probing questions, and offers observations. He doesn't "push" a viewpoint or agenda on the designer. It's up to the designer to execute their vision.

Tim doesn't dictate a resolution. In other episodes, it is extremely rare for Tim to make a direct suggestion for what to do with the design. In this clip, I observe the designer answering questions with an implied "question mark" at the end of his answer, possibly to elicit approval or direction from Tim. But Tim doesn't respond to that. He doesn't force an idea.

Through his probing questions and observations, Tim lets the other person explore options and find their own way. This clip is one example of that. I also tried to find another great example from the show, but for the life of me I can't find it on YouTube. In that coaching moment, Tim helped a designer who felt "stuck" on his design. In response, Tim suggested they look at the design from a different angle. "You are literally too close to this." So they went to the other end of the work room and looked at the design on the mannequin. Having some sense of separation helped the designer, who immediately saw the problem and had an idea for what to do.

When we provide coaching, we need to avoid pushing our own agenda onto the discussion. Let the other person explore new ideas, let them make the discovery. If you think you know the "answer" and offer your own solution, then whatever they do is your "fault." They won't feel any ownership in the decision. Instead, ask questions that allow the person to work through a problem and work out the issue on their own terms. When they return to work, they will have ownership in what they are doing.

Thursday, December 15, 2016

Leadership lessons from unusual places: My Little Pony

I'm continuing my recap of leadership lessons from unusual places, from one of my presentations at the Government IT Symposium last week.

In this leadership lesson, we can learn about building relationships from My Little Pony Friendship is Magic. What a wonderful show! It's full of lessons about how to build and maintain relationships. In fact, the whole show is nothing but about how to build and maintain relationships. Because friendship really is magic.

Relationships are the currency to getting things done. Think about how many times you have been stuck on something, and you've needed to call someone for help? As a manager or director, I'm sure you've had to call another work group or another division and say, "I'm having a problem working through X group, can you give me a hand here?" or "I'm trying to do X, but I'm getting stuck, can you help out?" or "Looks like I made a mistake here, can you help me smooth things over?" I think most of us have had conversations like that more than once.

The key in those conversations is relationships. You know someone who can help, and you reach out to them. Because you have an existing relationship with that other person, they feel a connection and are motivated to help. And you can really only ask those favors from people who you know well. You can't ask that of people you don't really know, or people you just met. If someone you don't know just came up to you and asked, "I'm trying to work with this other group, but their manager is stonewalling me, can you help me get around that?" then that would be weird. You don't know the person who is asking the favor, so it's odd that they would ask you for help.

Think about your social network. I like to imagine it like a bullseye target, where the closer you are to the center, the "closer" your relationship to me. The center circle is the "circle of trust," the people you might go to for completely confidential advice. These are the people you might ask for help if you were looking for a new job. The next circle contains those people who would help you with a favor. Outside that is the "parking orbit," people who are not very close to you, but with whom you are friendly; you might see them in the hallway or by the elevator, but not interact with them very much. And if you aren't in any of those circles, I call them "potential new friends," people I haven't met yet.


You can arrange your social network even further. Think of who are your personal friends, versus your friends at work. Who are your mentors, the people you look to for inspiration? And who are your peers, people with whom you interact but who are neither "personal" nor "work" friends?

You need relationships to get things done. Relationships are that important.

But how do you build relationships? Just remember the four I's of relationships:

  1. Initiate
  2. Inquire
  3. Invest
  4. Inspire

Start by meeting a new person, and reaching out to them (Initiate). Start looking for connections by asking questions (Inquire) and getting to know the other person. Over time, as you become closer, you Invest time in your relationship. This builds bonds. Eventually, you may find you can use your relationship to Inspire the other person to do great things.

Let's go to My Little Pony to observe the first two I's of relationships: Initiate and Inquire.



Here, you can see Twilight Sparkle using Initiate ("Hello. My name's Twilight Sparkle") to start the conversation, then Inquire ("What's your name?") to prompt the other person. She also asks follow-up questions to get to know the other person. Despite Fluttershy's introverted tendencies, Twilight Sparkle reaches out to get to know the new pony, making sure she heard the name right, and commenting on Fluttershy's birds in the tree.

Twilight Sparkle only has time for the first two steps. The third step, Invest, will happen over time as Twilight Sparkle continues to renew her friendship with Fluttershy through activities and adventures. That's just part of the show. Over time, Twilight Sparkle can rely on that relationship to inspire Fluttershy to do great things. That's also part of the show.

You can use the same method of Initiate, Inquire, Invest, Inspire to build your own relationship networks. The more people you know, the better you can navigate your organization and get things done. But don't let your relationships grow stale; fins opportunities to renew your friendships. If you call from someone in your relationship network, take a few moments to catch up before getting down to the task at hand. Or simply call or visit that other person, just to say hi and see what's up. These short moments help to build up your relationship currency.

Wednesday, December 14, 2016

Leadership lessons from unusual places: Disney's Mulan

Continuing my recap from last week's Government IT Symposium, I wanted to share a few insights from my presentation about "leadership lessons from unusual places."

One other resource I shared was Disney's Mulan. This is an entertaining animated feature about one woman's brave decision and personal journey to protect her father and—at the same time—save China from the invading Shan Yu. It really is a wonderful film and I don't want to distract from the message of the movie or the performance of the actors.

But when I watched Mulan, I immediately realized the leadership qualities of Shan Yu. It turns out that Shan Yu is very adept at coaching and mentoring. You can see this demonstrated in several moments throughout the film, whenever he interacts with his warriors. This scene is particularly instructive:



Watch that clip carefully and you'll learn a few tips on coaching from Master Shan Yu.

Note how Shan Yu uses this opportune moment to coach his staff. He finds a doll, and asks his team leads for what they can learn by examining the doll. They each provide their own insight that fills in a larger picture: the doll comes from a village high in the mountains, and the Imperial cannon brigade is there too.

As the viewer, we don't know whether or not Shan Yu knew this ahead of time. If he does, Shan Yu is careful not to insert his own point of view before asking the opinions of his team. He simply asks, "What do you see?" and lets others fill in the blanks.

Shan Yu uses this "coaching moment" to engage his team. His comments are brief, memorable, but not overpowering. Shan Yu is able to offer his own opinion (and decision to return the doll) without discounting the opinions of his team leads. From what we see in the movie, I think we can assume Shan Yu has taken advantage of other coaching moments to help his future leaders develop.

I like to refer to these "coaching moments" as "coaching buttons." These are the moments before and after a meeting when you find yourself chatting with one other person on your team. Or that moment in an elevator when it's just you and a staff member. Take these moments as gifts, and leverage them to do coaching. "Coaching buttons" are wonderful conversational gifts. The "coaching button" might only cover one question without an opportunity for follow-up questions to delve deeper. But if you can find frequent opportunities for several "buttons", I find it can be helpful.

Be like Shan Yu. Use these moments wisely and turn them into coaching opportunities.

Tuesday, December 13, 2016

Leadership lessons from unusual places: Pulp Fiction

At last week's Government IT Symposium, I talked about leadership lessons from unusual places. I wanted to share another source for learning about leadership. Specifically, leading small group discussions.

Let's watch this brief clip from the movie Pulp Fiction. This is from the beginning of the movie when Jules (Samuel L. Jackson) and Vincent (John Travolta) meet with some business partners of their employer. It's a meeting over breakfast.



In the Symposium presentation, we examined this clip through a new lens. Don't look at it as "gangster meeting with thugs," but rather "manager meeting with business partners." What are some lessons you can take away from this scene? I think you can learn a few things about leading a small group discussion.

Find commonality. Note how Jules immediately makes a connection with others in the room. He is clearly an extrovert, and can find opportunities to make small talk. But you don't have to be an extrovert to do this. Follow Jules's example to make small talk, and put others at ease.

Lead with clarity. Early in the conversation, Jules asks a few clarifying questions to set context and make sure everyone is aware of the purpose for the discussion. "Do you know who we are?" and "You do remember your business partner, don't you?" serve as a kind of introduction and help people to get on the same page.

Observe cultural norms. I used to work in higher education, as campus CIO for a small university. I remember that a key part of our campus culture was based on food. It seemed everyone brought food to meetings. It was just something you did. So I started to bring food to meetings, too. This helped to fit the cultural norm. It also helped to smooth over difficult meetings. So if you're going into a meeting to discuss a difficult topic, and especially if that meeting is first-thing in the morning, bring some doughnuts or muffins. Having food can help put people at ease.

Get into character. Did you notice what Jules said to Vincent at the very beginning of the clip? "Let's get into character." A colleague of mine often says, "Leadership is a performance." Sometimes you need to assume the character traits you want others to see. Do you need others to see you as collected and calm? Observe your body language and what messages you send when you attend your next meeting. How you project yourself says a lot about you; use that. Another colleague of mine advises to pause for fifteen seconds before you walk into a meeting. This is especially important if you are a few minutes late for the meeting. Another few seconds won't make a difference, but how you appear in the meeting will. If you appear unnerved (because you are late to the meeting) that may change the tone of the discussion that follows. So take a few moments to "get into character" before you join the meeting.

Monday, December 12, 2016

Leadership lessons from unusual places: Breaking Bad

I wanted to share another set of leadership lessons from my presentation at the Government IT Symposium.

I like to find leadership lessons from a variety of places. I think it's more interesting to look for leadership lessons in unusual places. If you look for leadership lessons from expected places, it's boring. If someone came you to you and said "It just watched that movie about Steve Jobs, and I learned that to lead a technology company you need to have passion and be driven to innovate," then you would reply "Of course, that's the point of the movie." That's boring. Instead, I like to look for leadership lessons in unusual places.

Let's take the show Breaking Bad. It was a very popular show, and if you haven't seen it, I recommend you do. In brief, the show is about an instructor who leaves academia, meets a former student, and gives remedial chemistry lessons to him while they entrepreneurially create a small business which they then grow. I think that's a fair description of the show.

And if you've seen the show, I hope that helps you to look at it from a fresh perspective. That will help in finding leadership lessons.

Let's examine a few key characters from the show:
  1. Walter
  2. Gus
  3. Hank
  4. Hector

While the show has a lot of characters, these four exhibit some great leadership traits.
Walter
When Walter leaves academia, what's the first thing he does? He teams up with Jessie. Walt knew he couldn't go it alone. Sure, Walt had the technical background they needed, but Jessie had already established several relationships that they would need to bring their product to market. Everyone needs to be part of a team.

And that's true whether you're working on an IT project, coordinating a campus-wide effort, or cooking a batch of meth in a dilapidated RV in the deserts of New Mexico.

While it's important to partner with someone who works well with you, be mindful that you select someone who brings a fresh perspective. If your partner carries the same opinions that you do, you'll fail to identify issues before they become problems, and you'll miss valuable opportunities.
Gus
The photo is from the first time that Gus meets Walter. It's a key moment in the show. And what can you say about how Gus approached Walter? I think you'll agree that Gus was very careful. Gus wanted to be sure Walter was someone he could work with, and that Walter wanted to work with him.

A colleague once related to me that hiring new people onto a team is a half-million dollar decision, and you need to treat it as such. Think of it this way: in higher ed and in industry, expect to spend about $100,000 per IT professional per year in salary and fringe (in most metro areas). Hire the wrong person, and IT managers may spend several years trying to "fix" the person they hired before deciding to end the relationship. And in the meantime, your team will experience "collateral damage" as more experienced members try to repair mistakes caused by the problem person.

Be like Gus. Whether he's hiring a fry cook in his fast-food restaurant or a partner in a new business venture, he builds a solid understanding of who he's dealing with. Large or small, Gus is careful to hire only the right people for his team.
Hank
Throughout the series, Hank is trying to catch bad guys. The photo shows Hank in his "war room" where he mapped out connections in an effort to identify the ringleader of an underground group. One thing you can say about Hank is he was meticulous in his planning. He examined details closely. And this paid off for him later, when he finally connected the dots to make a great discovery.

Take a lesson from Hank. Take care to plan out your vision and clarify your team's role in executing that vision. Apply proper methodology to your project planning, such as detailed effort work plans or simple Gantt charts, to ensure a smooth delivery. Map out any obstacles in your way and get people together to identify how to work around them.

But leave some flexibility in your planning. There is such a thing as becoming too focused. Avoid becoming overly attached to an idea so that it prevents you from seeing other possibilities and alternative outcomes.
Hector
I think we can all agree that Hector was a determined man. He carefully weighed his decisions, and having decided on a direction, he committed himself. I selected this photo of Hector because it shows him while he is making one of these important decisions. Walter comes to Hector with a proposal, and asks for Hector's help in getting out of trouble. Walter doesn't have a prior relationship with Hector, but he is persuasive nonetheless. Hector considers the offer, and is "all in." Because of Hector's commitment to the agreement, the season ends with a bang.

When you make a decision or decide on a course of action, there shouldn't be any doubt that it's the right thing to do. Balance your options, and weigh the benefits against the risks. And when you move forward, put all your energy into it. If you hold back, you aren't really committed.

Look at Hector. When Walter approached him with an opportunity, no one could argue he didn't commit himself to the deal. He evaluated the options, and was all in. Be like Hector. Ring the bell of success.

Saturday, December 10, 2016

Leadership lessons from unusual places: Star Wars

At Symposium this year, I gave one of my favorite talks: leadership lessons from unusual places. I sometimes like to find leadership lessons in unusual places. Looking for leadership lessons through the lens of unexpected sources can be interesting and insightful. In this session, we learned about leadership from various sources including: Disney's Mulan (coaching), Breaking Bad (commitment, hiring), Star Wars (taking on a new role) and My Little Pony Friendship is Magic (relationships).

While I've written about these leadership lessons elsewhere in my blog, I'd like to share them again here.
Let's start with my original example: Star Wars: Return of the Jedi. When I first looked for leadership lessons from unusual places, I started with Star Wars as my example. Pretty much everyone has seen Star Wars, so it makes for a good starting point. We all have that common ground.

But as we look for leadership lessons here, I need you to use a different lens. Don't look at Star Wars as space opera. Rather, look at it as an example of leading through change.

First, you have the Emperor. But before we can discuss him, we have a "news flash." It appears the Emperor has died in an industrial accident. While touring the "peace moon" known as the Digitally Enhanced Array, Troop Habitation, Satellite Transmitter And Receiver (aka "DEATHSTAR") the Emperor has met his untimely end. We don't know much at this time, but we can assume leadership will move down to the next "rung" on the ladder: Darth Vader.

Vader has a very similar leadership style to the Emperor. He's callous, officious, and sometimes micro-managing.

Now assume the persona of someone providing advice to Vader during this difficult transition in leadership. What advice would you give Vader to help him in his first 100 days?

In my Symposium session, I asked the audience to share their advice. People recommended Vader communicate his vision to the admirals and other leadership within the Empire. Understand the organization, learn how they operate. Maybe do a SWOT exercise to identify focus areas. Overall, he should share his broad themes early.

But wait! We have another "news flash" from the forest moon of Endor. It turns out Darth Vader also perished with the Emperor, at the hands of Luke Skywalker. Effectively a coup, Skywalker will now assume the mantle of leadership. We don't know much about Skywalker, but as a Rebel we know that he is the polar opposite of Vader and the Emperor; we can assume his leadership style will be quite different, as well.

Consider your persona of someone providing advice to Skywalker during his transition to the seat of power. What advice could you give to Skywalker to help him take on this new leadership role?

At Symposium, I asked the audience to give their advice. People responded that Luke should meet with the existing leadership to help them understand who he is, and what he wants to get done. Talk about his leadership style. Communicate broadly, and build relationships. Overall, people recommended that Skywalker share themes about his transition so others knew what to expect.

Consider the recommendations. Read them again.

People gave the same recommendations for leading through change regardless of leadership style. We expect Vader and Skywalker to have very different leadership styles, but how they lead through change is the same:

  • Communicate your vision
  • Understand the organization
  • Identify focus areas
  • Build relationships
  • Share broad themes early

It doesn't matter what end of the leadership spectrum you are at, you should follow the same guidelines when leading through any transition.

Friday, December 9, 2016

Disaster Recovery Planning at Symposium

This year, I was proud to be part of the Minnesota Government IT Symposium. One of my roles as a committee member was leading an all-day disaster recovery and business continuity planning "tech jam" workshop. Thanks to Bill Bushey, Kelly Clausen, and Cat Beltmann for helping to plan and execute the tech jam.

About twenty IT professionals joined us on Tuesday to compare notes about disasters, discuss disaster recovery planning, and to design architectures to respond to different disasters. On Wednesday, I shared what we learned in a 90-minute "report out" presentation during Symposium. I'd like to share a brief summary of that here.

First, what is disaster recovery compared to business continuity?

In the simplest definition: disaster recovery is what you do to recover your system from a disaster. Business continuity is what you do to keep conducting business while others recover the system.

In most organizations, IT focuses on the disaster recovery, and the business units focus on the business continuity.

When we talk about disaster recovery, most people immediately think of the "smoking hole in the ground" scenario. They imagine examples where the data center building is engulfed in an inferno or demolished by a tornado. In fact, disasters can be large or small.

In the workshop, we led a small group breakout to discuss different disasters we had all experienced. We heard examples like a database administrator not checking the status of a backup before deleting a database to upgrade the database software (in fact, the database backup had failed). Or of building management conducting a fire alarm system test, and accidentally shutting off all power to the data center. Or a water leak from an overhead sprinkler system that damaged several racks of systems in a data center. Or construction workers who accidentally cut into the only fiber data connection for the data center.

So a disaster recovery plan needs to accommodate different types of disasters, different scales from large to small. In the best case, organizations can design an architecture for their systems that remains flexible to an outage. The best disaster recovery plan is one that doesn't require people to get involved. But if you aren't able to have systems automatically "fail over" during an outage, you want to have a plan documented beforehand about how to recover your systems.

Folks divided into four working groups, and spent the rest of the day designing an architecture and documenting a response plan for different scenarios. One group discussed how to recover from partial data center damage, and another group prepared a plan for a new property tax system. A third group designed an architecture to extend flexibility for a network in the face of a possible ISP outage, while a fourth group used the workshop to outline an actual disaster recovery plan.

For details from the four working groups, please see the materials we shared from the Symposium. We captured the work of all the working groups and have made the information available to Symposium attendees. We encourage you to use this as a starting point for disaster recovery planning in your own IT organizations. Let this work help you get a jump start on planning for disasters.

At a high level, here are several key "take-aways" from the four working groups:
1. Partial data center damage
Identify critical people and vendors.

In a disaster, you will need to immediately reach out to technology folks to respond to the disaster, and vendors to help you get back to business quickly. Your next disaster may not be conveniently timed to occur during working hours. Make sure you have copies of contact information available off-site.
Network is important for business.

With so many organizations using Cloud and vendor-hosted applications, the network is key to your business. If your partial data center outage impacts the network, your users may not be able to access these remote systems. Also consider authentication and other components to your network.
Maintain a warm site.

If possible, you should implement dual "hot" sites where you run production applications and systems from two or more sites.
Identify critical systems and applications.
Conduct a risk analysis or other determination to understand which systems are truly your most critical. Create disaster recovery plans for these systems first. For example, most organizations can let "development" or "test" systems wait a few days while you bring up production systems.
2. Property tax system outage
Are your RTO and RPO realistic and achievable?
How quickly can you recover a system, and how old will your data be when you get it back up? these address two important factors in disaster recovery planning: RTO (Recovery Time Objective) is how long it will take to recover your system, and RPO (Recovery Point Objective) describes the "age" of the data that you can recover.
Understand the impact.
What is the cost of an outage? In this scenario, the working group was able to quantify some numbers. For example, the interest on $100 million over three days. Use the cost of an outage to help you justify additional infrastructure.
BC and DR can sometimes intersect.
In most instances, disaster recovery is performed by the IT team, and business continuity is the responsibility of business units. But depending on the application, note that the "players" for these may overlap.
Design for redundancy and failover.
Where possible, create an architecture that remains flexible in the face of an outage. In one example, you might run production systems from two different data centers. But as you design "failover" into your architecture, look for single points of failure and address them.
3. Network and ISP
Network is business critical.
This was a repeated theme throughout the day. Many organizations now outsource applications, and leverage Cloud and vendor-hosted systems. While this simplifies your IT, it means the network becomes even more critical to your operations.
Identify single points of failure.
Another repeated theme from the workshop. As you design your architecture, consider what systems are dependent on other systems. Watch for any single points of failure.
What is your upstream's DR plan?
Be mindful not to be lulled into a sense of security when outsourcing applications and services. Just because you have outsourced part of your systems to a vendor or other upstream provider doesn't mean you can ignore disaster recovery planning. Discuss disaster recovery plans with your upstream vendor to understand how they will bring your applications and data back online in the face of a disaster.
4. Disaster recovery planning
Identify systems and inter-dependencies.
As you create a disaster recovery plan for an application, look closely at the systems it connects to. What inter-dependencies exist? How does your application rely on other systems and applications?
Document the architecture.
You can advance a disaster recovery plan very quickly simply by documenting the architecture. Make this a group exercise, to sketch how each component of an application runs on different systems, and how those systems are connected.
How do you go back to normal?
It's tempting to focus only on the actual "recovery" portion of a disaster recovery plan. For example, your plan may require moving production to a "test" server. But you can't run production on the secondary system forever. Having "recovered" on the test system, how do you plan to go back to a normal state?
Analyze risk to find critical systems.
Use a risk analysis method to identify which systems and applications are the most critical to your business. One way to do this is to break it up into components: the likelihood of a failure, and the business impact of the failure. The combination of these components determines the criticality of the overall system.
I thought it was great that two of our working groups used Tuesday's workshop to carry forward work that is currently underway in their organizations. If you are a local CIO who sent staff to this workshop, your teams came away with actionable results.