I had success reducing the amount of memory taken up by ng build by changing some settings in angular.json. These are the ones that reduced memory:

vendorChunk: false
optimization: true

then removed the sourceMap option, which should stop Angular from supplying sourcemaps for vendor files