Have you ever came across a job posting advertisement that was an EXACT match for your background?
You meet every single requirement and nearly all of the preferred qualifications.  Maybe you even get an interview...
You go for the interview.  You build rapport with the Hiring Manager, get them to smile a few times, and you proved to them why you're a perfect fit based on the job description and requirements they advertised.  Everything seemed to go very well and you're excited about the opportunity!

Then a week goes by, and you haven't heard anything.  You call the recruiter to follow up, and they tell you that you are still being considered, but the Hiring Manager is "still interviewing other candidates" or hasn't made a decision yet.  You ask when you can expect a decision.  The recruiter tells you probably by late next week.  Next week Wednesday rolls around and the Recruiter calls you (if you're lucky, maybe you just receive a templated rejection email) and vaguely explains that the hiring manager just decided to select another candidate with qualifications more closely matching specifically what they were looking for.

As a Career Coach, I've heard this same story many times. In fact, throughout my Recruiting/HR career, I've experienced it personally while searching for a job myself several times. Since I work as a Recruiter, I know what happens behind the scenes. There are many possibilities why the Hiring Manager did NOT select YOU. 

First of all, you need to think back and recall if you remembered to ask the following question: 
(ideally the Recruiter before the interview and/or to the Hiring Manager as early as possible in the interview)

“What would you say are the top 3 most important characteristics or qualifications that you’re looking for in a candidate for this role and why?”

The reason is that, many times, job postings usually are not 100% accurate to what the Hiring Manager specifically wants. Recruiting & Human Resources Departments have been squeezed to maximum capacity over the last 5-8 years because they are "overhead" to the organization. Typically, their time is very limited and being stretched in many directions at once.  When the Recruiter that is responsible intakes the information about what the Hiring Manager wants, they first start with a job description provided by HR that has been recycled from a similar role. Then, the details are tweaked. There are a million questions that the Recruiter could ask the Hiring Manager, but they gather the main attributes. Sometimes the Hiring Manager changes their mind, forgets to mention something, or didn't realize they needed something until after the fact.

Many industries & professions have very specific differentiations in environment where experience in one segment could be VERY different in another segment, even though the responsibility could be described the same way. It's difficult to conceptualize, so let's discuss a couple examples:
  • Recruiting - There are essentially 3 different types of environments that one could be working in:  Staffing Agency / Independent (Selling candidates to various client companies for a fee), Corporate / HR / Talent Acquisition (Recruiting for 1 company, acting as part of the HR Dept.), or RPO - Recruitment Process Outsourcing (Representing part or all of a particular client company's Recruiting team exclusively).  While many of the skills required are the same, there are also some differences in the business relationship and how to handle different situations. Each setting is unique in how the interests of all parties compete for profit:  candidate, hiring company, and staffing company.
  • Clinical Research - You could be working as a Clinical Trial Manager in 3 different settings within the pharmaceutical industry:  Sponsor company (Company that develops and manufactures the drug), CRO - Contract Research Organization (Company that outsources certain aspects of a clinical study), or ARO - Academic Research Organization (College University's Research Dept. that provides its students with projects to work on at their medical research center, while providing valuable data to the Sponsor company).  All 3 settings require similar responsibilities of a Clinical Trial Manager, however, the scope of responsibility is very different in each where some may handle the full process from start to finish, while others handle only part of the process, and may not even be aware of what happens outside of their personal experience.

Those examples are all within the SAME industry.  Now, imagine when candidates want to switch to a similar job title in a completely different industry. You will be considered much less desireable in most cases.

Is a lot of the experience transferable?  YES. 
Are there certain things that you can ONLY learn by gaining experience directly from each of those different business settings?  YES.

In the tight economy these days, Hiring Managers are already overworked, trying to get by with less staff, and have been asking their Manager for approval to add headcount to their Dept. for months.  In many cases, they just can't justify the risk of a bad hire that isn't ready to "hit the ground running" and would require training specific to their business environment.

***  Beginning of rant ***
Is it fair?  NO.  But, it is the way that it is because the Hiring Manager's head will be on the chopping block if their department fails.  Companies have an obligation to make a profit for their stockholders or business owner(s) that invested their own money, which allowed the company to operate, exist, and grow in the first place.  Employees and job seekers at each company need to understand and be thankful for the company providing the opportunity of employment to earn a steady paycheck!

Of course there is a moral/ethical balance required between company profit and fairness to the employees that make the company profit, so there are always two sides to the arguement. The ONLY alternative to avoid the "unfairness" is to untake the risk of starting your own company!  Then you can run it HOWEVER YOU WANT.  After you pour your blood, sweat, and tears into starting your own business, you may feel that you deserve to be a little bit greedy because you've earned it.
*** End of rant ***

There are many other possible reasons why you could be rejected even though you fit the description perfectly, but that is one of the top reasons and is mostly beyond your control.  The important thing to understand is that you need to figure out quickly what exactly the ideal background they are looking for is.  Do your best to highlight how you meet the described qualifications and how you could be successful at little risk to them.  Lastly, do not take it personally if they select someone else.