We present a simple phenomenological within-host model describing both the interaction between a pathogen and the immune system and the waning of immunity after clearing of the pathogen. We implement the model into a Bayesian hierarchical framework to estimate its parameters for pertussis using Markov chain Monte Carlo methods. We identify a threshold antibody level that separates a large increase in antibody level upon infection from a small increase and accordingly might be interpreted as a threshold separating clinical from subclinical infections. To study the effects of immune structure on population level prevalence we implement the within-host model at the individual level as a building block into a population level transmission model. For the case of very short infections and a constant force of infection this leads to a relatively simple linear first order partial differential equation. Using a generation expansion we compute the age-dependent ratio of the probability of observing a symptomatic infection in the individual host in a next infection event to the probability of observing an asymptomatic infection.