Also don't use arm arch if not defined

This *should* fix build so precise, but I didn't fire one off.
I did test that builds with libseccomp2 still work as expected.
Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
Acked-by: Stéphane Graber <stgraber@ubuntu.com>