This is a nice little paper that provides a great introduction to machine teaching. I think the Socratic dialogue format was an excellent choice as it makes it very easy to follow.
The big problem with machine teaching in many practical applications is what the paper refers to as the "glaring flaw", and that is that you often don't know what the learning algorithm might look like (e.g. in the provided nefarious example of trying to defeat a spam filter). In fact, the learning algorithm could be arbitrarily complex.
In the case where you do know the learning algorithm exactly (e.g. the learner is a robot where you have its precise specifications), the problem is the deterministic optimization problem described in this paper. But when the learning algorithm is unknown, the problem becomes stochastic, and then you're facing all of the traditional problems with optimization in a probabilistic space (e.g. overfitting, robustness problems, etc). That's not to say that it's strictly impossible to apply machine teaching approaches in such a case, it's just that it's a much more difficult problem to find a somewhat optimal training set.
The big problem with machine teaching in many practical applications is what the paper refers to as the "glaring flaw", and that is that you often don't know what the learning algorithm might look like (e.g. in the provided nefarious example of trying to defeat a spam filter). In fact, the learning algorithm could be arbitrarily complex.
In the case where you do know the learning algorithm exactly (e.g. the learner is a robot where you have its precise specifications), the problem is the deterministic optimization problem described in this paper. But when the learning algorithm is unknown, the problem becomes stochastic, and then you're facing all of the traditional problems with optimization in a probabilistic space (e.g. overfitting, robustness problems, etc). That's not to say that it's strictly impossible to apply machine teaching approaches in such a case, it's just that it's a much more difficult problem to find a somewhat optimal training set.