Lessons learned looking for deer

Lesson 1: Cover your face

Over the summer, our family was out camping, and we noticed a small herd of deer wandering through the campground. Our three kids figured they'd practice some stalking (they were pretty bored), so they put on their camo shirts and pants, and snuck off towards the three does and the forked-horn buck that were grazing about 100 yards from our campsite. The results were predictable - the deer were tame enough to let them get within about 20 yards, but too smart to let them get closer.

But afterwards, our eldest decided that he wanted to practice stalking me. The game was that I would pretend to just be staring off into space, occasionally looking around, and if anything he was doing was obvious, if I saw him making a clear movement or if something about his presence jumped out at me, he had to start over.

And I actually learned something while being stalked by an 11-year-old that I hadn't expected: his face and hands really showed up. He tried sneaking in on his belly, and from 20 yards I couldn't distinguish his camo clothes at all. But his pale face and bare hands were almost luminescent compared to the dark foliage around him, and were by far the most noticeable part of his body.

That was when I bought all three of us ghilly hats and camo gloves.







The hats look pretty ridiculous, and they fog up your glasses, but they hide your face well, break up your form, and maybe just as important, they keep out the mosquitos. (They're almost worth it just for that last.)

Lesson 2: Stop moving

When you spot a deer, the very, very first thing you want to do is simple: stop moving. Period. Freeze. Don't move a muscle, until you figure out what's going on and have made a plan.

Surprisingly, this can be harder said than done. My first instinct is apparently to shrink back behind the nearest cover. But that movement into hiding attracts a deer's attention, and has cost me several opportunities. In contrast, when I've just stayed still, even when the deer totally had me, it generally didn't spook.

When I was hunting down in southern Oregon this last September, Chris Zornes and I were approaching a watering hole where we were hoping to setup for the evening. We were early enough that we weren't expecting to find any deer there, but about a hundred yards out, we spotted five or six deer milling around the hole. They were all does and fawns, so far as we could tell, but we still didn't want to spook them off. We stayed at a distance until they appeared to leave, then moved closer.

About 40 yards out from the hole, downwind, I was moving from one clump of brush to another, when two does and a fawn suddenly walked out from some brush near the hole, and I was caught totally in the open, moving. This time, at least, I managed to freeze, and went motionless. The lead doe caught something, and stared at me, but didn't spook. When she looked away for a moment, I moved slowly to put a small tree between us. It wouldn't hide me complely, but at least it would break up my form. The three deer stayed there for several minutes before the lead doe finally got suspicious and cut a circle around behind us, to get downwind. She was about 10-15 yards away when she finally got a whiff of our scent, and then she (and the rest of the herd) bolted.

The next day, I didn't exercise as much control, and ended up blowing a stalk. I'd jumped a doe and a small buck, but the buck stopped about 80 yards away on a hillside, and after a few minutes, I watched him settle down under a tree. He had an excellent view of the country around him, but after some thought, I was able to work out a path that would keep me mostly downwind and mostly screened to within 30 yards, which is about my range for an ethical shot. So I got down on my belly, and started crawling. I'd closed to 50 yards, checking periodically to make sure he hadn't moved, when I had to cross an open area. There was some brush blocking his view of that opening, so I started slowly crawling forward. Several feet out, I glanced his direction, and realized that he'd stood up and moved a few yards. From this new vantage point, his view wasn't blocked at all, and he was staring right at me. Without thinking, I scooted back behind the nearest bush - maybe the dumbest thing I could have done. He looked at me for five seconds, and then disappeared up the hill. I don't know if freezing would have kept him from getting spooked, but pulling back certainly put the icing on the cake.

Lesson 3: Look for the other deer

When you spot a deer, before you do anything, look for the other deer around it. Usually deer aren't traveling by themselves, so when you see one, there are probably more. It's surprisingly easy to get so focused on the one you're looking at that you miss the others standing a few feet away.

This happened to me the only time (so far) that I've actually taken a shot at a deer. It was in a spike-or-better unit down in Oregon, and on our fourth day of hunting, Chris and I saw a small but legal deer standing a short ways off the road. We stopped his truck, and the deer didn't move. I felt a little awkward about shooting a spike - he didn't have spots, but he wasn't much more than a largish fawn. But it would have been my first deer, and I wasn't about to turn it down. So I got out of the truck, eased out my bow, and ranged him. 20 yards. This was going to be easy.

Then he started to move away. I wasn't sure how far, but he stopped after a few yards, and I guessed he was maybe at 25. I didn't want to take the time to re-range, so I put his shoulder blades between my 20 and 30 yard pins, and let fly. I couldn't quite tell where the arrow went, but he jumped and kicked, and ran about 10 yards before stopping and looking back.

I immediately grabbed for another arrow, to see if I could get a follow-up shot. Right as I nocked it, I finally noticed that he was being followed by another doe, and a much larger fork. They had been standing there in plain sight this whole time, and Chris and I had both totally missed them. Not being sure whether my first arrow had gone home, I didn't want to take a shot at the fork, so I ignored him, and tried to maneuver for a second shot on the spike. But while I was trying to get into position, they decided to move - quickly this time - and were gone.

We eventually found my arrow, and it was completely clean - no blood on it at all. Our best guess is that he was closer than I'd believed, and that I shot over his back: it must have touched him just enough to give him the willies and make him jump. But if I'd kept my eyes open, I would have seen the fork, and given their direction of travel, probably would have had time to range him correctly.

Lesson 4: Screen your form

When you're looking through the forest for deer, by far the easiest thing to see is movement. After that, it's something distinctive, usually their ears, sometimes their tail. But if the deer is motionless, in the shadows, and they're screened by some brush, they're surprisingly hard to see.

The only benefit is that this can also work in our favor. Animal brains recognize movement, and they recognize form, so when you move slowly (it has to be very slowly), and break up your form, you can make it much harder for them to realize there's a human standing there. When stalking a deer, if you keep something between you and your quarry - a rock, a tree trunk, a small bush - even if you're partially visible, you have a much better chance that the deer will look right past you.

To drive this point home, I recommend that you try sneaking up on someone in the woods sometime. (Not your wife. Not if yours is anything like mine and you don't enjoy sleeping on the couch.)

Down in Oregon, Chris and I had split up to do some still hunting - he was going to take one ridge, and I was going to take the other. An hour or so later, I saw that he'd crossed over onto my ridge, and was walking up towards me, maybe two hundred yards away. He sat down to glass, and I decided to see how close I could get before he spotted me. I knew it couldn't be very close, because I had to cross a whole bunch of open area, but I figured I'd give it a shot.

I approached him from the side, walking slowly, in the open sunlight. There was a small tree about 15 feet to his right, and I put that between us. As I walked he glanced a time or two in my direction, and I thought for sure he'd spot me, but he gave no sign of recognition, so I kept moving slowly, putting my feet down as quietly as I could. After 10 minutes or so, I was standing just 15 feet from him, behind the tree, and I had all the reward I could hope for when I stepped from behind the tree and watched him jump half out of his skin.



Absent a buddy who's not paying attention, try sometime to see how close you can sit to a well-traveled trail, without the hikers seeing you. I've had folks out scouting pass 10 yards from where I was sitting, lightly screened, without noticing I was there. (I've also been the guy feeling the surge of adrenaline when I realize there's another hunter sitting only a short distance away.)

Lesson 5: Watch your silhouette

I was still hunting one sunny morning when I spotted a deer's silhouette moving on the far side of some brush about 40 yards away, followed shortly by two more. I was out in the open, but in the shade, and I figured that the same brush screening them would screen me, so I slowly moved towards the only opening where they could come out. Several minutes passed, and they never appeared - and I never did see them again.

Only afterwards, when I was thinking about it, did I realize that even though (a) there was brush between me and the deer, and (b) I was in the shade, right behind me was a bright, golden, sunny field. Even through the brush, my moving and shaded silhouette would have stood out like a drag queen at church.

Lesson 6: Noise disappears faster than scent

Yeah, yeah, if you've read this far, you know this, right? So did I. But it's one thing to know it in theory, and another to actually watch deer react to your smell, while you're watching them.

Chris and I had decided to setup at a watering hole in southern Oregon - the same watering hole I mentioned above. We parked his truck about a quarter mile away, and walked in along the main trail, with the idea that it would be quieter than approaching through the brush on the other side. I expressed some concern that the deer coming in on this trail would hit our scent, but Chris (who has been hunting the area for close to 15 years) said, "Nah, I don't think that they pay too much attention to human scent around here. It's just too common."

We got close to the hole, and split up. Chris camped out on a hillside slightly above the hole, and I followed a deer trail up a small draw, moved about 15 yards out, and setup in some brush, well hidden.

An hour later, I watched a small herd of does come down another trail towards the hole. They were upwind of my position, but about sixty yards away, they hit our back trail, and every one of them immediately stopped. They milled around for about five minutes, looking for the human predators they could obviously smell, before deciding that a drink wasn't worth it, and they bailed.

About 15 minutes later, another herd came down the draw. They passed within 15 yards of me, still upwind, but then the lead doe hit my back trail. She nearly jumped out of her skin. She looked around desperately, trying to spot me, and then bailed. Oddly enough, the other three deer didn't immediately follow her. They milled around for a few minutes, afraid to go further, but reluctant to leave. One of the fawns actually put its nose down on the path I had taken, and began to follow it right up to where I was sitting. About then, the lead doe started wheezing loudly, from about a hundred yards back up the draw. It was a sound I had never heard a deer make before, and I can only imagine that she was warning them, insisting that they follow her and get out. They were obviously thirsty, because they didn't want to leave the hole without a drink, but eventually they took her warning, and disappeared. None of them ever spotted me, but there was no doubt they knew I was there.

We didn't see any bucks that night, but the lesson was clear: we should have cared more about whether a deer would smell us on the main trail, and less about whether they would have heard us in the brush. The noise would have been long gone by the time we were situated; but the scent lasted.

A grand unified theory of hunting

I mentioned in my last post that animals are much better at knowing where we are than we are at knowing where they are. But we can also extend that thought slightly, as it presents the primary dilemma associated with hunting.

(1) If you want to see animals, you need to move.
(2) If you move, animals will see you.

All the different hunting techniques are just attempts to overcome this paradox.

Another way of looking at this is to say that hunting is an attempt to maximize your daily "encounter zone" (the daily acreage where you can detect a deer) while minimizing your "warning zone" (the daily acreage where a deer can detect you). Deer are better than you are at this, so your warning zone is normally bigger than your encounter zone. The real bugger, though is that most of what you can do to increase your encounter zone is going to increase your warning zone; and nearly every technique that decreases your warning zone is also going to decrease your encounter zone.

The idea that you have to be moving to see deer is basically just math, though it's empirically verifiable as well. If you pick some random spot out in the woods, and sit there, you're not going to see many animals. You're much more likely to see a deer if you engage in some counter-intuitive behaviors, like driving your truck slowly along logging roads, or even better, riding your mountain bike.

The reason is that deer don't move much during the day, and when they do, it's slowly. During daylight hours, a blacktail might only walk a quarter mile. (It'll move further at night, but that doesn't help a hunter.) If you can see that deer from 50 yards, the territory from which it's visible that day will amount to about 10 acres (440 yards * 100 yards = 44,000 square yards = 9.1 acres). In other words, there's only a 10 acre "encounter zone" with a randomly positioned stationary hunter, and your chances of your location intersecting with the deer's encounter zone is relatively small.

But if you're out mountain biking, and you cover 10 miles with the same 50 yards of visibility, you'll have put 330 acres of territory within your purview, so you've got a much larger encounter zone. Mathematically, there's simply a better chance that a deer will be in those 330 acres than that a stationary hunter's location will fall within the 10 acres from which a traveling deer can be seen. But because you're moving, and because deer are better at these things than we are, any deer will be able to tell you're passing by from probably 100 yards away - so your warning zone will probably extend to over 600 acres.



This is also why so many hunters are primarily road hunters. You really do see more animals that way. Over the last two seasons, I've  laid eyes on 150 or so deer. Probably 100 of those were from a moving vehicle of some sort (auto, truck or bike), and maybe 40 were from when I was walking or still hunting. I've only seen maybe a dozen deer when I was setup somewhere, silent and quiet - despite having spent much of my time in the woods completely motionless.

Me, motionless

The problem is that when you're moving, the deer also see you. This goes back to how animals (including the two-legged variety) interpret their visual field. Long ago, evolution figured out that things which move are more relevant to our survival than things which don't, so our brains are hard-wired to pay attention to movement. When you're moving, you scatter more scent around; and you make more noise. So yeah, you see the deer, but all you see of them is black tails and white butts, disappearing into the brush.

So what do you do? Well, like I said, every hunting technique is just an attempt to get around the "10 acre problem". But each of them in turn introduces its own set of complications.

Still hunting - walking very slowly and quietly - is an attempt to minimize your warning zone by keeping signs of your presence below the deer's "excitement threshold", while still covering enough territory to increase your encounter zone to something reasonable. This is probably my favorite sort of hunting, because you're doing something that requires skill and concentration, and your view is constantly changing.

Still hunting in southern Oregon

The downside is that a deer's excitement threshold is quite low. It's hard - harder than you would think - to keep a deer from seeing you if you're not absolutely motionless. Sound, scent and sight are all against you. I can only think of a few times, while still-hunting, that I was able to spot deer before they knew I was there, and it's no accident that they were all were in relatively arid southern Oregon. The forest we were hunting was open enough that I could see further than the deer could hear, and when the wind was right, I sometimes got lucky enough to notice the deer first. (Most of the time, of course, by the time I noticed them, they were either staring straight at me, or were disappearing through the tree trunks.) I've never managed to accomplish this in the much denser and louder undergrowth of western Washington. If you want to be quiet anywhere west of the Cascades, you have to be on a road or well-traveled trail - which is usually open enough that the deer are going to spot your unscreened movements well before you take any account of their screened, still, grey bodies. Or you end up traveling so slowly that you don't really open up your encounter zone very much. There's probably some sweet spot, but I haven't found it yet.

Stand hunting - positioning yourself in a tree stand or a ground blind and staying put - is an attempt to put yourself within the right 10 acres. You make your encounter zone huge during the scouting season, letting you make an educated guess as to where the deer are likely to be come hunting season. Your encounter zone while actually hunting is tiny, but assuming you remain still, quiet and motionless, your warning zone should be even tinier. (It will extend just a few yards upwind, and a quarter mile or so downwind.)

Setting up a tree stand

The main downside to this approach, if you're hunting western Washington blacktail, is that they're hippy deer. They go where they want, when they want, with no pattern I've been able to discern. I've been told you can set your clock with whitetails - if you see them nibbling a particular patch of alfalfa at 7:00 am today, you've got a pretty good chance of find them there at 7:00 am tomorrow. No self-respecting blacktail would ever let themselves get tied down to a schedule like that. They go anywhere and everywhere. It's not uncommon for a well-marked trail to get used maybe a couple times a week. That's a lot of time to be sitting in a tree stand.

The books will tell you, by the way, that blacktail do have certain patterns: that there are differences between feeding areas, daytime bedding areas, and night-time bedding areas; between trails used for traveling to feeding areas, those used traveling away from feeding areas, or those used for escape. These books say that with careful study, you can figure out which is which, and predict where blacktail are going to be.

Unfortunately, blacktail haven't read those books.

Spot-and-stalk hunting - trying to spot animals from a distance and then sneak up on them - is an attempt to increase your encounter zone via binoculars and careful positioning. Once you've spotted an animal and know where it is, you can focus your movements, and remain careful enough during your stalk to (help) ensure that the animal stays outside your warning zone.

The main downside to this approach is, "Have you ever been in western Washington?" This is what it looks like:




Or this:



Or this:



Yeah, good luck spotting and stalking with that.

To do reasonable "spot and stalk", I think you need to be in a place where you can see at least 10-20% of the land around you at 400 yards.

This is maybe a little thick for spot-and-stalk, but it's at least plausible

Most deer will still be hidden, but you've got a fighting chance. In most wild places in western Washington, you're gonna be lucky if you can see that much of the land around you at 50 yards. Some places, it's gonna be 5. There are locations where maybe you can try it - big, recent clearcuts - but you're restricted to glassing the edges of the cuts, which is pretty limiting. I've never seen a western Washington blacktail in a clearcut open enough to glass from a distance.

Road hunting basically says, "Screw it, I'm gonna make both my encounter and my warning zones so huge that if I get lucky, a deer might get confused and let me have a shot." I have a feeling that when deer hear a truck or a mountain bike coming down a logging road, or even a noisy hiker on a trail, they often conclude, "Nothing making that much noise can possibly be dangerous." Or maybe it takes them 10 seconds to make up their mind - but by that time, you're traveling fast enough that you've moved within the encounter zone.

A deer I saw while walking, which spooked as soon as I stopped to take its picture

The biggest downside to moving quickly is that the deer are always going to know you're there, and you're only sometimes going to know that they are. Deer hide really well - more on that in a future post - and in a western Washington forest, they're almost invisible until they move. If you're moving quickly, you're unlikely to see a screened, motionless deer. (Hell, they can be hard enough to see when you know they're there, and you've been staring at the spot for five minutes.) When the deer finally does move, it'll be heading quickly towards brush or other landscape features that it knows will screen its movements, and it's unlikely you'll get a shot off in time.

I should note that if you're in a vehicle and see some deer beside the road, if you keep going and don't slow down, the deer will often let you pass without moving. But if you stop, or try to get out and off the road, the deer will be gone in seconds. The only exception to this that I've seen is if the deer is a yearling without its mother. Then it'll sometimes remain still. The one actual shot I've taken at a deer - and blew - was of this sort.

Oddly enough, if you're moving slowly, but with just enough movement or sound that the deer can tell you're coming, they're more likely to spook than if you're moving quickly and noisily. I once watched my kids trying to stalk some deer at a campground. So long as the deer knew where they were, my kids could get fairly close: the deer didn't really care about all the noise and commotion. But when my kids tried to circle back some distance and approach from behind, the deer quickly locked in on the sounds of unknown origin, and bolted. Deer really don't like anything that gives the impression of trying to sneak up on them.

So what do you?

I don't know yet. Nothing I've tried has worked. Yet. But I've learned a lot about the details, and I'll have some comments on what precisely I've picked up in future posts.

Why hunt?

One of the things I don't think I realized when I started hunting was just how much better animals are at hiding than humans are at finding them.

When I started hunting last year, some church friends gave me permission to hunt on 40 acres of their property. Their parcel is in a fairly rural area, and was recently logged. It's on a slope that backs up against a steep hill and provides access to thousands of acres of public land. It's a gorgeous hillside, with a perfect mix of deep timber, recent clear cuts, deciduous trees and evergreens, and lots of blackberries.



I've since spent most of my time in the woods focused on scouting and hunting that area - hundreds of hours, in all seasons, in all weather patterns, and using every technique I can think of.

The reason why is that there are animals there. There are bear and cougar and bobcat and coyotes.






It's full of deer beds, deer browse, and trees that the bucks have been knocking the snot out of.





There are enormous tracks and deep trails and huge piles of scat.





I've got three trail cameras out there, and I've been able to identify at least six different bucks, from tiny spikes up to really nice five points. Given typical buck:doe ratios, if there's that many identifiable bucks, it means that there are probably 30 or more deer somewhere on that hillside.

It's a rich, rich area.

And in all that time, I've seen exactly four deer.

Four deer. That's all. And all does.

Only once would it have been legal for me to take a shot, and they were gone before I could think. I never once drew back my bow with an animal in my sights.

The only way I can square what I'm seeing - the sign and the trail camera pics - and what I'm not seeing - actual animals - is that blacktail are really, really smart. I've come to the conclusion that if you’re in the woods, it’s generally a safe assumption that every animal for a quarter mile around knows exactly where you are.

Hunting is like playing hide and seek in a 27 million square foot house, with hiding places everywhere, trying to find someone who runs faster than Usain Bolt, hides better than John Rambo, smells better than a bloodhound, and moves as quietly as the grass grows. And who knows that his life depends on you never seeing him.

It's frustrating as hell.

And yet it's been an amazing experience. I’ve been a hiker and backpacker all my life, and have spent a lot of time outdoors, but I’ve never paid attention to the forest the way that I do now. Have you ever seen a billboard from a long distance off, and you can’t quite figure out what the picture is of? It’s just a blob of colors. But then you get closer, and suddenly it resolves itself into a recognizable image, and now you can’t not see it.

It’s that way for me with hunting. The woods were all one kind of big blob for me before. Now, I see signs of animals everywhere. I see where they’ve been walking. I see where they’ve been eating. I see where they’ve been sleeping, hiding, fighting and courting. I’m seeing the forest in three dimensions finally. I’m inside the woods in a way that I never was when I was just walking through them. If you’re a gardener, you know how different it feels to eat food you’ve grown yourself instead of just bought at the grocery store. It’s something like that: it turns out that the natural world has a texture and history, a structure and substance, that I had never understood.

When you’re sitting in a stand, the whole world feels different. You’re silent and still. You hear everything: the guy starting up his chainsaw two miles away, woodpeckers across the draw, a Douglas squirrel yelling at who knows what, crows arguing, leaves falling. You see birds flitting through the bushes, squirrels in every tree, chipmunks skittering across your boot. You know exactly where your scent is blowing, you notice every gust of wind, and grimace when it changes direction. But you’re not asleep, or even tired, because every moment you’re living in hope, you’re scanning the treeline, looking through the brush, listening with everything you can, waiting for that one moment - the one that in your heart of hearts you know you’re going to blow - when the deer you’ve been waiting for shows his head.

Some thoughts on blacktail hunting (from a total noob)

About three years ago, I bought my kids a couple of toy bows - some of those $30 jobs that you can find at any sporting goods store, which pull maybe 15-20 pounds, and shoot about as straight as Liberace. We couldn't hit anything with them, but it was fun plinking around. As a kid myself, I had borrowed the tiny wooden longbow my own Dad had as a kid, and loved it, but I think it came with maybe 1-2 arrows, and I quickly lost those. So this was my first chance to actually experience anything like archery.

After a year or two, I was still enjoying it enough that I wondered what it would be like to shoot a real bow - something dangerous enough to actually warrant the anxiety Galena had been lavishing on the toy bows. I knew that real archers actually sometimes hit what they were aiming at, and I was happy to blame my repeated failures in that regard on my equipment.

So after some research, and on a weekend when Galena wasn't paying too much attention, I found myself walking out of our local bow store with a new Hoyt PowerMax and all the fixings.



Not surprisingly, I found myself actually able to hit stuff. Not at all well, mind you - I was shooting something like 8" groupings at 20 yards - but way better than anything I'd been able to do with the kids' bows. And I loved the thunk that the arrows made sinking into a target at high speed. It was cool.

I didn't know enough to know that I shouldn't be impressed at this grouping

After a month or so of that, I started wanting to shoot at more than just targets in the backyard. Which got me thinking about hunting.

I was pushing fifty, and hadn't hunted since a family friend took me a couple times in high school. But I remembered the feeling of excitement and power that came from walking through the woods with a weapon in my hand. And I figured that if I was going to try for a mid-life crisis, this was about as innocent a pastime as Galena could ask for.

The first problem is that I really had no idea where to start. I was a complete newbie - I couldn't tell you the difference between a blacktail, a whitetail and a mulie, and had no idea which ones lived anywhere near me. None of my friends hunted, and none of the hunters in my extended family lived close enough. Just reading the Washington state hunting regulations was an exercise in frustration. But I started watching Youtube videos, reading hunting websites, and hanging out on hunting forums, and that got me hooked.

In one sense, those resources were invaluable. I had no idea, for instance, that if you shoot an animal with a bow, you're not supposed to jump up and run after it. I'd always thought that an animal shot with an arrow would act roughly the way a human in a movie does, and fall down with a dramatic thump. It wouldn't have crossed my mind to think that they would nearly always simply run away, like they'd never been hit. And I was a bit surprised at how seriously the hunters on the videos treated the wind. I had some vague theoretical notion that animals had a better sense of smell than humans, but I had never thought to pay attention to which way the wind was blowing.

In another sense, though, the resources weren't all that helpful, for several reasons. First, the vast majority of deer hunting resources are targeted towards whitetail hunters. I haven't yet had a chance to go after a whitetail, but after two seasons of hunting Columbian blacktails, I can tell you that they ain't nothing like their eastern cousins. I'll get into the details in later posts, but suffice to say that techniques which work well on whitetails are nearly irrelevant to the sort of hunting available in western Washington. Second, and this is just what you'd expect, the videos are nearly always about successful hunts. At best they'll mention the months of scouting and the fruitless weeks spent in a treestand, or even throw them into a montage, but that's not what they show. And that gives you a very different feel for what hunting feels like when you're actually doing it, especially in an area where deer don't travel in herds, spend nearly all their time in thick undergrowth, and are unlikely to ever be seen in significant numbers.

So I've been thinking that I'd like to throw up a few blog posts about my experiences as a total newcomer to the sport. It should be obvious that these are not going to be the authoritative pronouncements of an experienced hunter. (I've been hunting for one and a half seasons, haven't harvested anything bigger than a rabbit, and have taken a single shot at a deer - a spike - and missed clean. I'm in no position to be giving advice to anyone.) It's mostly a way for me to collect my own thoughts, and maybe help other newbie hunters who were in the same spot I was. And if you happen to be reading this, and can spot something I'm doing wrong or have totally missed - please, chime in!

The value and the limits of unit testing

Swyfft’s website (https://swyfft.com) is now live – for a certain definition of “live”, in this case, meaning the coastal counties of Alabama. Alabama is where we hope to get some good initial sales metrics, before we expand out to more states.

When I joined Swyfft back in April, depending on how you looked at it, the website was maybe 90% done, and maybe only about 10% done. I’m sure you’ve heard the old developers’ saying, “The first 90% of a project takes 90% of the time. The last 10% takes the other 90%.” And that was certainly true here.

In addition to adding more features, one of the biggest things we’ve been working on since April is automating our testing process. My strong preference, whenever it’s feasible, is to rely heavily on pure unit tests, with all dependencies mocked out. It’s easier to isolate the code under test when you do it that way, it’s easier to get the tests running on your build server, and the tests themselves run a lot faster.

Unfortunately, that approach has limited utility at Swyfft. The main reason is that our system relies heavily on over a million rows of meta-data, spread over 20 or 30 tables. Many of our most critical tests can be summarized as, “Does this code interact correctly with this metadata?” And by definition, that’s the sort of thing you can’t do with pure unit tests.

Consequently, I’ve split up our automated tests into roughly three categories:

1. Unit tests. These don’t talk to the real database, and any dependencies are strictly mocked out (we use Moq for this).
   
2. Integration tests. The entry point for these tests can be anywhere from the API controller in the website, down to the lowest-level repository, but the key is that they actually talk to the real database – though other dependencies (such as Authorize.net, our payment processor, or IMS, our agency management system) are generally still mocked out.
   
3. Acceptance tests. The line between these and integration tests is a little fuzzy sometimes, as we still use the same testing framework for both (we recently switched from MSTest to xUnit). But basically, the idea is that we don’t mock anything out, and each test exercises the system from front-to-back, usually in defined flows. We use Selenium to drive a web browser for tests that involve the website.

This works. Mostly.

Don’t get me wrong – our testing infrastructure is critical to Swyfft, and we couldn’t have gotten where we are without it. The thousand or so tests that we run on each build are an invaluable safety net. But there are plenty of things we’re going to need to improve. A few of them:

1. Selenium is susceptible to random failures. About half of our test runs will trigger at least one unrepeatable Selenium test failure. We’ve tried different browser drivers through Selenium, and Chrome is maybe the least problematic, but every one I’ve tested seems to have similar problems, just to different degrees. Those random failures significantly decrease the value of having a build automation server: just because Jenkins rejects the build doesn’t mean anything if you know that there’s a 50% chance the build was rejected because of something that isn’t really a problem in your code.
   
2. The whole thing is really slow. I prefer my test automation suite to run in under 60 seconds. Anymore than that, and running the tests become something you do when you think you’re done coding, rather than something you do repeatedly as you’re ginning up a new feature or performing a quick refactor. And unfortunately, our tests take about 15 minutes to run on a fast machine. You can work around the slowness, and we do, but it really interferes with getting realtime feedback.
   
3. Keeping the DB in sync on our build automation server is a PITA. We could regenerate the DB for each run, but it takes about 15 minutes or so just to load the metadata, and the test runs are long enough already.
   
4. We’ve done almost nothing about JavaScript unit testing. The site is simple enough so far that we can get by with UI-level automation testing, but as the site gets more and more complex, that’s not going to be sufficient. We could probably get rid of half of our problematic Selenium tests if we were able to verify the JavaScript (or rather, TypeScript) code at one or two levels lower down the stack.

I’d be interested in hearing how other folks have solved these problems. Please let me know!

Swyfft is off and running

For a little over a year, Swyfft, the company I joined last April, has been working to revolutionize homeowners' insurance. And we've got some seriously cool stuff going. Most of it I can't talk about - it's full of algorithms and analytics and patented this-and-that. But the result is straightforward enough: you go to our website, put in an address, and two or three seconds later, get a quote. I think we're the first insurance company to pull this off, and it's very cool.

Of course, a fast quote wouldn't be terribly helpful if it didn't also save people money. And we've got tricks up our sleeve there, too. Naturally, we can't save everyone money: claims are what they are. But Sean Maher, our very smart, nay, brilliant CEO (I'm not above bootlicking), has worked some magic there as well. And we think we can assess the sorts of risks a property is likely to face as well as or better than insurance companies who've had billions of dollars and decades trying.

Our systems, the piece that I'm most intimately involved with, have been more-or-less ready since late last summer. But the business side has taken quite a bit longer: negotiating the contracts, maneuvering through the regulations and licensing agencies, that sort of thing. But we're up and running now, and the response we've seen so far is promising and exciting.

Everyone has to start somewhere, and our initial market is small: coastal Alabama. But that's just the start. As we get initial metrics, and start to increase traction, we aim to be all over the Gulf Coast, and eventually, nationwide.

I love Swyfft, I love what I get to do every morning, and I can't wait to knock the socks off our industry.

Brief thoughts on Newman’s “Development of Doctrine”

On p. 47 of Newman’s An Essay on the Development of Christian Doctrine, Newman writes:

I have been arguing, in respect to the revealed doctrine, given to us from above in Christianity, first, that in consequence of its intellectual character, and as passing through the minds of so many generations of men, and as applied by them to so many purposes, and as investigated so curiously as to its capabilities, implicaitons, and bearings, it could not but grow or develop, as time went on, into a large theological system.

He’s correct on this point, I believe. Anything that impacts human beings is going to invoke human curiosity, and anything that claims to be as significant as Christianity will necessarily be the focus of a great deal of intellectual endeavor. I can’t imagine being human without some desire for consistency and coherency in what we believe, and I can’t imagine a human civilization without people who are focused on figuring out how each part of our social and intellectual worlds connects up to the other parts. If something big happens to humans – boom, humans are gonna start thinking about it. And arguing. And those arguments are going to get more sophisticated and subtle as time goes on, as flaws in earlier formulations become apparent, as various and sundry attempts are made to resolve the inevitable difficulties.
But Newman goes on to write:

…next, that if development must be, then, whereas Revelation is a heavenly gift, He who gave it virtually has not given it, unless He has also secured it from perversion and corruption, in all such development as comes upon it by the necessity of its nature, or, in other words, that that intellectual action through successive generations, which is the organ of development, must, so far forth as it can claim to have been put in t charge of the Revelation, be in its determinations infallible.

He’s wrong on this point, I think. He would be correct of the intellectual content of Christianity were its chief or most important characteristic. But if Christianity is the remnant and the witness to God having done something, and what God has done is more important than what we think or understand about what God did, then I can’t see any reason why God would find it necessary to preserve it from all perverson and corruption. As I understand Christianity, I can see how it would be important to shield it from a certain amount of perversion and corruption: what God is doing involves, among other things, making sure people learn about what God has already done. And there is an intellectual component to this. But it’s certainly not the only and probably not the most important component. Jesus said that all men would know that we were His disciples by our love for each other – and not, let us be clear, by the correctness of our theology. If that’s the case, I could see God being willing to put up with quite a lot of theological misunderstanding and wrongheadedness, for the sake of His people learning to be the sort of people He wants.