It depends what kind of algorythms will you use, not of the engine. UnrealEngine have the best tools ready to go for large scenes, but it isn't for 1GB ram.

I would suggest using Unity3D as it is easy to learn, you should also use such things like Octree and fast rendering methods like those: https://docs.unity3d.com/6000.0/Documentation/ScriptReference/Graphics.RenderMesh.html https://docs.unity3d.com/6000.0/Documentation/ScriptReference/Graphics.RenderMeshInstanced.html