Would this be an infringement of copyright law?
Unless it met the fair use/ fair dealing criteria of the country of publication then, yes, this is copyright violation. Fair use/fair dealing is jurisdiction specific and depends on the extent of copying and the use to which it is put. Note that US fair use is about the most liberal in the world: other nations are much more restrictive.
... who gives permission for something like this ...?
The owner of the copyright - this is usually the producer of the video and is usually stated with the copyright notice at the end of the credits.
Please note that the participants will commonly have moral copyright in most jurisdictions. This is not a proprietorial right but includes the right for attribution and integrity in the work (i.e. not to have their work dealt with in a way that is disparaging). A direct translation is unlikely to violate this but if the subject is highly controversial then you need to be very careful of language nuances that may not translate and that may subtly change the meaning of what was said; particularly if it is to be distributed without the original audio and visuals.