[PMC Weekly Consulting Insight] At last, some survey marketing conclusions

Philip Morgan


Finally, I've fixed my inconsistent coding of the survey responses from my LinkedIn sample, and have something to say about this little bit of research.

Let's start with the core question: how do self-employed software developers invest in their career?

Let's focus on the top 3 methods of investment, and differentiate between my two sample groups. Remember1, one group was a convenience opt-in sample2 from LinkedIn, and the other was a convenience opt-in sample from my email list.

Overall numbers

To paint a more complete picture, let me pull in some numbers I've previously mentioned about this survey and my two sample groups:

  • 22 people from LinkedIn completed the survey.
  • 33 people from my email list completed the survey.
  • The median age of respondents is 42 (LinkedIn sample and 41 (list sample).
  • 64% of the LinkedIn sample have invested in their career in some way. 36% have not.
  • 95% of the list sample have invested in their career in some way. 5% have not.

So as you're looking at the numbers below, keep these overall numbers in mind.

There are three sub-questions I'll focus on in this quick write-up:

1) How do you invest in technical skills?

The full question was: "Please list ways you have you spent time and money for developing your technical skills."

A response starting with "a-" is an action, while one starting with "s-" is a sentiment or belief. What you'll see below are the top 3 responses, and how many of my respondents indicated that action, sentiment, or belief, and what percentage of the total number of survey completions that number represents (rounded to the nearest whole number).

LinkedIn Sample:

  1. a-online-courses - 12 - 55%
  2. a-reading-books - 4 - 18%
  3. a-irl-conferences - 4 - 18%

List Sample:

  1. a-online-courses - 22 - 67%
  2. a-reading-books - 16 - 48%
  3. a-coding-practice - 9 - 27%

2) How do you invest in business skills?

The full question was: "Please list ways you have you spent time and money for business or self-employment skills."

LinkedIn Sample:

  1. a-reading-books - 2 - 9%
  2. a-learning - 2 - 9%
  3. a-experimenting - 2 - 9%

List Sample:

  1. a-online-courses - 15 - 45%
  2. a-reading-books - 15 - 45%
  3. a-email-lists - 6 - 18%

3) Where does opportunity come from?

The full question was: "Consider your entire career as a self-employed software developer and times you have gotten new opportunities, better projects, or other forms of career improvement. What do you think led to these improvements in your career?"

LinkedIn Sample

  1. a-networking - 6 - 27%
  2. s-perceptiveness - 3 - 14%
  3. s-persistence - 2 - 9%

List Sample

  1. a-networking - 8 - 24%
  2. a-relationships - 5 - 15%
  3. a-relationship-with-existing-clients - 4 - 12%


What conclusions can we draw from this slice of the data? Remember that I'm someone who is in the business of helping dev shops make smart investments in the owner's career capital, so my conclusions are filtered through a sort of meta question, which is: "What changes might Philip need to make to more effectively reach and serve the people he's focused on serving?"

I think the following are conclusions that are supported by this survey data:

1) Books and online courses are the best way to reach and help devs who are trying to invest in their career because those are the most common media that investment-minded devs seek out.

2) If you're helping devs acquire new technical skills, operate from a hands-on learning perspective rather than a theoretical perspective. 3

3) If you use an email list to reach people, your best method of growing your email list might be via other people's email list.

I'll expand on this conclusion a bit, because it might not be self-evident.

There was a stark difference between my two sample groups here. No LinkedIn respondents mentioned email lists (like this one you're probably reading this article via) as a method by which they would invest in their career. Six of my list respondents mentioned this method of investment, and it was listed both as a method for investing in technical skills and as a method for investing in business skills.

Of course, this is a clear sampling bias. If you ask enough members of an email list that focuses on investing in your career how they invest in their career, some of them are bound to call out... the email list by which you recruited them for the survey as one of the means of investment they use!

But it also suggests that if someone is on Email List A because they have a preference for that form of learning/investment, they might be very open to joining Email List B and Email List C and so on. They might be more open to doing this than the average person.

This suggests that if I am running Email List C, then finding a way for Email List A's owner to expose my email list to their list members could be an effective way for my email list to reach more people.

In a way, this is super simple, basic stuff. We all know it's easier and cheaper to sell again to an existing customer than it is to acquire a new customer. We all know it's easier to deepen and extend a customer's engagement with something than it is to introduce that thing to someone who has never considered it and has no context in which to understand or value it.

So of course it's easier to attract a new list subscriber if that person already values being on an email list! But also... I wasn't really thinking about other people's email lists as a way I could attract more people to my email list, despite this seeming so obvious when faced with my survey data.

I have tended to think of growing my email list in the following terms:

Get on podcasts as a guest -> end with a call to action to join an email course -> let the email course do the work of inviting people onto my email list

This works well, and yes, it does resemble a digital marketing funnel, albeit one where the work of trust-building is pushed towards the front of the funnel. Front-loading the trust-building is somewhat atypical for digital marketing funnels which often entirely ignore trust building and seem to focus only on filtering for and selling to those who are desperate.

My survey data suggests that I might want to invest in a second -- perhaps complementary -- funnel. One that has a different "mouth" that involves other people's email list.

Onward to my final conclusion:

4) Opportunity comes from relationships and market awareness.

As I often like to say: unfortunately, you're in a relationship business.

My survey data bears this out. :)

The final conclusion

My survey focused mostly on inputs (except for the final question, which asks respondents to speculate about the causal relationship between inputs and outcomes). For example, I asked "Please list ways you have you spent time and money for developing your technical skills." rather than "Please list the most effective methods you've used for developing your technical skills." or "Please list the ways of developing your technical skills that you've found to produce the best result for the least time spent." This seems like a flaw, or an important but overlooked part of answering my core question of "how do self-employed software developers invest in their career?". It's nice to know how they do this, but it would be even better to be able to say what produces the best ROI in terms of desirable career outcomes.

I'm now in a good position to get an answer to that question. I could design a second survey that uses my first survey (which I've referred to as the "de-biasing survey") to scope and fashion the questions in the followup survey, which would have this as its core question: "Please rank these 5 methods for improving your career in order of effectiveness."

I'm exaggerating only a little bit when I say I thought my initial survey's questions were nearly perfect when I sent them out. It was only when I started seeing the responses that I got ideas for improving them and a general realization that they were not perfect. "20-20 hindsight!" as they say.

This is what experiential learning looks like. You 1) take your best shot, 2) facilitate meaningful feedback from those who are not incentivized to protect your ego, 3) iterate, and 4) repeat.

I do believe this survey has shortened the feedback loop from 1 and 3. For me, that might be its primary value.

Next steps on this project are:

  1. Take what I wrote above and tweak it to be less about me and more about my survey respondents.
  2. Send this report to the respondents who asked for it.

I'm on the fence about whether to ask some of the respondents for an interview in order to enrich my data set or whether to call this "sprint" done and move on to the next research question, or a refined version of this question using my 20-20 hindsight. Either way, I'll take a break from this lengthy series and write about some other stuff that I've been experimenting with, most likely telling you about a sales/CRM experiment I've been running in my business.

I will end with this blatant pitch: I'm working on getting another Expertise Incubator group together. I need at least 3 people to start another group, and I cap each group at 5. I've got 2 folks who are in, so if you'd like to join a small group of folks who are pushing themselves to cultivate valuable expertise -- and who in the second quarter of the program engage in a research project like what I've described here -- then please hit REPLY and let me know. The cost is $2,100/quarter, and the program creates a lot of productive discomfort, so bear that in mind as you consider whether it's a fit for you. If you reply, I'll suggest we set up a short video call to figure out if the fit is ideal for where you are in your career.

Here's an absolutely bizarre, yet oddly enchanting song I ran across recently: https://youtu.be/m276i7UIyNw It's called "Taming the Dragon", by Brad Mehldau, and it's an offbeat spoken word thing with segments of heavy, trippy drumming and synthesizer layered in. Kind of like MMW's "Whatever Happened to Gus" and "Your Name is Snake Anthony".

Here's what's been happening on my paid Daily Consulting Insights email list:

  • Unless we're sociopaths, we maintain loyalties to people, groups, and ideas. Some of those are harmful.
  • What can you learn from superstar performers? Tim Ferriss thinks you can learn a lot. I'm not so sure.
  • Reality distortion fields: They are FOR SHIZZLE not the easiest way to make money. They are not "finding an existing river of money and dropping your fishing line into it". They're more like constructing the Panama Canal from scratch. Yet, some of us build our businesses this way. Why?


  1. If you want to read up on this experiment, check out what might be one of my longest -- and potentially most boring -- series of emails:
    1. philipmorganconsulting.com/pmc-survey-marketing/
    2. philipmorganconsulting.com/pmc-the-de-biasing-survey/
    3. philipmorganconsulting.com/pmc-survey-marketing-recruitment/
    4. philipmorganconsulting.com/pmc-survey-marketing-initial-data-from-the-de-biasing-survey/
    5. philipmorganconsulting.com/pmc-weekly-insight-survey-marketing-qualitative-analysis-of-the-de-biasing-survey/
    6. philipmorganconsulting.com/pmc-weekly-insight-survey-marketing-coding-and-counting/
    7. philipmorganconsulting.com/pmc-weekly-insight-its-a-grind-grind/
    8. philipmorganconsulting.com/pmc-weekly-insight-slogging-away-the-morning/
    9. philipmorganconsulting.com/pmc-weekly-insight-coding-and-altered-states/
  2. A convenience opt-in sample is where you invite a group of potential survey respondents to participate in the survey. You may or may not incentivize their participation, but you are definitely not using statistical methods to attempt to control for or compensate for sampling bias. In fact, that's the whole strength of convenience sampling methods: they're fast and cheap compared to probability sampling methods or census sampling methods.
    Of course, the lack of statistical sophistication is the main point some will criticize if you use convenience samples. I have a TEI participant who is engaged in an interesting conversation with one of his list members who seems to see this as a huge deal. His list member seems to be saying that since a convenience sampling method has uncontrolled bias, it's completely worthless.
    I disagree, because new information, even if it comes from a biased sample, can facilitate significant relative decreases in uncertainty, which can provide impressive value to a business decision maker. Furthermore, even if you're not using strict statistical methods to control or compensate for bias in a sample group, you can be aware of what biases might be present in a convenience sample, draw reasonable conclusions about how those biases might influence the information you gather from your sample, and incorporate that information into your decision making.
  3. Anyone who has ever tried to learn something about coding, or tried to teach others about it probably understands this intuitively, but it's nice to also see this reflected in the data.
    This also makes me feel more confident about my choice to teach business and marketing skills from an experiential learning perspective rather than a theoretical perspective. Just because the skills exist in a different domain doesn't mean I should use a different learning modality for teaching them.