When to choose one model strategy against the other:
- Computational speed: Conditional or Discriminative models need to observe all data to be optimally trained. So they fit poorly as far as computational efficiency is concerned. Usually Generative models learn each data instance in isolation, so they run faster.
- Complex business domain where Classes are inter-dependent: Since Conditional models observe all data they learn inter-dependency of Classes. Here Conditional models fit better.
- Evolving business domain where new Classes appear: Since each model is trained in isolation, Generative models fit better. In Discriminative models the whole system needs to get retrained.
- Missing data: Generative models consider relationship among variables equally, so in the case of missing data they degrade gracefully. Conditional models must be trained for the missing data.
- Corrupt data: Generative models can detect corrupt data and user can accordingly take corrective measures. Conditional models perform poorly.
- Quality of Inference: Better and quality algorithms like EM of Generative models outperform likes of Gradient descent of Conditional models.
- Feature selection: Conditional models ignore the features which don't contribute to the end result. Generative models can't distinguish between useful and useless features.
- Computational resources: As Conditional models are highly specialized and focus on specific features, they are better at resource utilization.
- Dealing with Test data: Since Testing is all about evaluating conditional likelihood, it is intuitive to realize that Conditional models fit better as they would have already optimized better on Training data.
However, these two approaches can be mixed based on various needs.
