Patch elf binaries rpath
You can use the binary patchelf, which must be installed for your architecture, like this:
patchelf --set-rpath '$$ORIGIN/../lib' <binary>
The double dollar sign is needed for escaping purposes
The patchelf may not be installed in your host, or may not be available for your architecture (esp in cross-compiling targets).
You can use the binary elfedit, which must be installed for your architecture, like this:
elfedit -e 'dyn:runpath $$ORIGIN/../lib' <binary>
The double dollar sign is needed for escaping purposes
When cross-compiling for armhf on amd64 in a debian-based distro you may want to install the package crossbuild-essential-armhf, which includes arm-linux-gnueabihf-elfedit