Supplying a hardware input frame which is not in the input hardware frames
context is not allowed by the API, so additional code to handle it is not
necessary.  Further, handling it automatically results in very low
performance - it is more appropriate to fail immediately so that the user
can fix their incorrect setup.