“Fire the Communicator”

A week ago, NTM-A/CSTC-A (the command I work for) sent a team to an Afghan village to investigate, I believe, allegations that coalition forces caused the deaths of several Afghan civilians. The team was headed by a two star General, and consisted of a small cadre of military members. The task to my branch (CJ-6 Operations) was to set this team up with a stand-alone network (self contained… not connected to the outside world) that would enable them to collaborate while they completed their due diligence.

A Navy Lieutenant Commander (0-4 equivalent to a Marine Major) heads up, among other things, my “CG’s Comm Team” – the team of military members responsible for ensuring that the Commanding General has ubiquitous communication capability. His first task was to recommend which communicator would accompany the team and set up this network upon arrival. Since the General assigned to this investigation had already completed one with a communicator that was still in the CJ-6, the recommendation was easy. The decision was made, coordination completed, and that communicator reported to the LCDR to begin preparing the equipment. The communicator knew exactly what was taken on the previous trip, so he simply stated his requirements, the LCDR initiated the tasking, and the communicator collected the equipment.

When the LCDR briefed me that everything was assembled and ready, I directed him to set everything up in the conference room and prep a demo for me to ensure that everything worked. Upon completion of the successful demo, the LCDR and I proceeded to brainstorm all of the redundancies we could build into the equipment set to ensure that the team could still operate in the event of catastrophic equipment failure… thumb drives for file transfers, extra batteries, cables, laptops, hard drives, ink cartridges for the printer, etc. To my relief, the LCDR and the communicator on the team had already covered these bases. Then it dawned on me that we only had one network switch to connect everything; so I told the LCDR to round up another one.

After an exhaustive search, he came back to tell me that we had no spares, which came as a shock to me. With only one week on the ground at the time, I assumed that we would’ve had a stockpile of comm equipment for the myriad of similar missions that we are tasked with. I made a mental note to review our outstanding equipment orders; and to have my staff put the paperwork together for any future requirements that were not already on order. I told the LCDR that we couldn’t send this team out without an extra hub or switch, to which he replied that the mini-exchange (retail store) on the compound sells a small hub for $40 that could serve as a back-up. I gave him the $40 and told him to buy it. He came back a few minutes later and handed me my money back, saying that the hub was too small to support all the devices that would be on the network. Disappointed, I tasked him with double-checking with every communicator on the compound to see if one would surface. Unfortunately, it was already too late to send someone to another base to pick one up.

I thought to myself as I sent Tony off on this wild goose chase, “Is it really that big of a deal? What are the odds that the switch, of all things, would fail on this five day (or so) trip?” Then I recalled an occasion in Iraq where the unthinkable happened… I was dual-hatted as Headquarters and Service Battalion Executive Officer and Communication Company Commander. On one particular day, one of my Comm Company leaders came to me and said, “Sir, both of our generators are about to fail” (he used a different phrase, but I’ll err on the side of political correctness here). We had two 350Kw commercial generators pushing more power than we could possibly use on our compound; but we were in the desert (for the engineers reading this, no, the generators weren’t wet stacking); and the generators we were using, as well maintained as we kept them (at the time, the brother – great guy - of the Commandant of the Marine Corps was our generator maintainer; but that’s another story), were not equipped for the 24/7 dusty grind that we were putting them through.

Since the Battalion Commander was travelling, and I was his second in command, I happened to be the Battalion Commander on that particular day. Coincidentally, there happened to be a Commanding General’s (CG’s) staff meeting that morning; and, because I was “acting” Battalion Commander, I had a seat at the table and a briefing role. When my turn came, I said, “Sir, you are about to lose all communications throughout the Al Anbar Province.” He said, “What’s going on?”. When I briefed him on the situation with the generators, he said, “What do you need?”. “Triple redundancy, sir. I need another 350Kw generator”, I replied, wondering if that was really the correct term – if you have two backups, is that considered double redundancy or triple? But Marine speak seemed most appropriate, so I went with triple because it has the number three and I needed a total of three generators.

The CG looked at the Engineer Battalion Commander and said, “Make it happen, Joe”. I was in awe of the trust the CG had placed in my advice; and at the speed with which he made the decision and was able to effect the change. In a very short time, another huge 350 watt generator was plopped onto the Comm Company compound; and, as the engineers were connecting it, the unthinkable happened… both of the existing generators failed. I wondered if the engineers made this happen for effect, but quickly dismissed the thought as I realized that the 2nd Marine Logistics Group had just lost all network connectivity to the outside world; and we had convoys out all over the province. Fortunately, the total outage was no more than fifteen minutes, since all of the pieces were in place, and the engineers were in the final stages of connecting the third generator… a few minutes of downtime for something that could have taken hours or even days, thanks to the moral courage and foresight of my Marines; and to the decisiveness of the General and the quick response of the Engineer Battalion.

The lessons I learned that day were forever etched in my memory. So yes, there must be an extra switch… we cannot send this team out without one. I didn’t want to be responsible for any delays this team might face, potentially keeping them in harm’s way long enough for a suicide bomber to kill them all. Sure, we included thumb drives; and the team could have used them to transfer files between the computers, but it would cost them time that I didn’t think they had.

Tony returned with no luck. I knew by the look on his face that he had done everything he could. “Here, go buy the hub”, I said. Tony and I knew that the hub was the next best redundant solution. The hub was purchased, the gear packed up, and the team left… fire and forget. We then went on with our mission as the team of investigators headed off on theirs.

Last night, as I returned from a terrific Q&A session that the CNO (Chief of Naval Operations) and MCPON (Master Chief Petty Officer of the Navy) held with the Marines and Sailors of NTM-A/CSTC-A (the subject of a future BLOG), imagine my surprise when Tony said, “Sir, the switch failed on the last day of the investigation.” “Damn”, I thought, we failed them… forgetting in that instant that we actually did have the base covered. Tony smiled and said, “They pulled out the hub and were back on line in a minute." He then proceeded to tell me how the team had to leave the printer unplugged until they needed to print, at which time one of them would have to give up their network connectivity so another could print; but they still had full network capability. What a relief.

I still don’t know how the General perceived the failure of equipment and rapid recovery – he may have been pissed that he couldn’t print immediately; but that little hub represented a fairly sizeable victory for the CJ-6 ops branch. Unfortunately in our profession, victory comes in the form of transparency… if comm is up, nobody notices and very seldom is the team praised for such success. When comm goes down, everybody notices and criticisms abound.

Brigadier General Craparotta, Assistant Division Commander of the Second Marine Division, and a Marine for whom I have great respect, has a three part saying for fixing communication problems – wisdom that he imparted upon me during a staff meeting when I was his G-6 (senior communicator) after an important Video Teleconference (VTC) between him and the Commanding General (his boss) in the desert had just failed. I don’t remember the first two lines; but the last one resonates with me… “Fire the communicator”.

Not today boss…