Doctors are not omniscient. They miss things. Some are sedentary, and they don't understand how important a full recovery is to an athletic person.
Everyone, when going to a doc, needs to think through what to ask his doc, and know how to ask -- or insist -- effectively. We are not giving the OP medical advice here; we are suggesting how to manage his visits to his docs effectively.
I suggest:
(1) if you suspect concussion, drop everything and go to a doc, an ER or a walk-in clinic sooner than ASAP.
(2) When you visit your docs, bring someone with you who knows your situation well, is outspoken without being abrasive, and can advocate for you. Such a person is called a Patient Advocate and, according to the Institute for Health Care Improvement can be a professional PA or a family member or close friend. The Institute says:
A time of illness is a stressful time for patients as well as for
their families. The best-laid plans can go awry, judgment is impaired,
and, put simply, you are not at your best when you are sick. Patients
need someone who can look out for their best interests and help
navigate the confusing healthcare system–in other words, an
advocate.....
An advocate is a “supporter, believer, sponsor, promoter, campaigner,
backer, or spokesperson.”...... An effective advocate is someone you
trust who is willing to act on your behalf as well as someone who can
work well with other members of your healthcare team such as your
doctors and nurses.
An advocate may be a member of your family, such as a spouse, a child,
another family member, or a close friend. Another type of advocate is
a professional advocate. Hospitals usually have professionals who play
this role called Patient Representatives or Patient Advocates.