Task forces for everything

I had a discussion a while ago with a manager senior to me. I wanted to solve a problem by putting together a team of the most suitable people to deliver a highly coveted result. Based on my understanding of Conway's law this would be the most effective way to solve the problem. Not only would it greatly reduce the administrative overhead, it would also likely be the only way to achieve the result.

I was met with the comment that putting together task forces had been tried before and would for sure be successful, but it would not be a way to operate at a larger scale. I was left with the feeling that although there might be some truth in this there where some flaws to the argument:

1. Even if creating task forces to achieve valuable goals is not a scalable strategy, for sure we could do more of it until we get diminishing returns.
2. If the assumption is that the candidates for working in a task force are always the same people from a small group, rather than picking persons from that existing group we should expand the number of people that would qualify for this group.
3. The thought that members of a task force are some kind of "best of the best" is bad thinking. Members of a task force should be the best people for the task at hand, there is no general best of the best for a task force in general.

So in true agile fashion when the principles are sound, try to leverage it a little bit more. Can task forces be used for everything, is it possible to constantly reorganise to optimize for the number of people working in task forces solving high value problems?

Note: Task forces sometimes have a negative impression by usually being created to repair issues. This tends to be the worst part of being in a task force, you fix problems caused by others, usually at great time pressure and stress. If you remove the bad parts, you'll probably find a lot of skilled talent interested in being a part of a task force.