How to convert an Xcode project that uses automatic build to a command-line project using make?

Question

I've been working on my game engine, which is for mac on Xcode.

However, I'm a little bit tired of the complex configuration in Xcode. Terminal based projects do not have this problem. I can simply type make to compile my game.

I wonder how to do this. My project is a Swift + Metal project, which is like this:

.
├── ***
│   ├── aabb.swift
│   ├── bvh.swift
│   ├── camera.swift
│   ├── imageProcess.swift
│   ├── interval.swift
│   ├── loadScene.swift
│   ├── quad.swift
│   ├── ray.swift
│   ├── sphere.swift
│   ├── texture.swift
│   ├── triangle.swift
│   └── vec3.swift
├── ***
│   ├── aabb.metal
│   ├── camera.metal
│   ├── color.metal
│   ├── headers
│   │   ├── aabb.metal
│   │   ├── color.metal
│   │   ├── hittable.metal
│   │   ├── interval.metal
│   │   ├── material.metal
│   │   ├── quad.metal
│   │   ├── ray.metal
│   │   ├── sphere.metal
│   │   ├── texture.metal
│   │   ├── triangle.metal
│   │   └── vec3.metal
│   ├── hittable.metal
│   ├── interval.metal
│   ├── material.metal
│   ├── quad.metal
│   ├── ray.metal
│   ├── sphere.metal
│   ├── texture.metal
│   ├── triangle.metal
│   └── vec3.metal
├── main.swift
├── output.ppm
├── resources
...
└── types.h (Swift bridging header, also included in Metal)

13 directories, 69 files

I know how to compile swift (swiftc -c ~.swift -o ~.o) and how to compile Metal (`metal -c ~.metal -o ~.air) but I just don't know how to link them together to a single binary file.

It is a Metal program so I may have to build a metallib file. This is my main.swift file I wish it could help:

import Foundation
import Metal
var renderSetting = renderSettings()
renderSetting.imageWidth = 1000
renderSetting.imageRatio = 1.0;
renderSetting.VFOV = 90;
renderSetting.defocusAngle = 0;
renderSetting.focusDistance = 10.0;
renderSetting.samplePerPixel = 5;
renderSetting.maxTracingDepth = 2;
renderSetting.lookFrom = createNewVector(3, -5, 3);
renderSetting.lookAt = createNewVector(0, -4.5, 4);
renderSetting.vup = createNewVector(0, 1, 0);
renderSetting.backGroundColor = createNewVector(1, 1, 1);

var objectSet: [object] = []
var objectSetIndex = 0

var imageTextureSet: [UInt8] = []
var imageTextureIndex = 0

loadScene(mapName: "gammaTestingScene", renderSetting: &renderSetting,
          objectSet: &objectSet, objectSetIndex: &objectSetIndex,
          imageTextureSet: &imageTextureSet, imageTextureIndex: &imageTextureIndex)

initialize(renderSetting: &renderSetting)

var sceneSet: [scene] = []
var sceneIndex: Int32 = 0
_ = buildBVH(objects: objectSet, start: 0, end: Int(renderSetting.objectAmount), sceneSet: &sceneSet, index: &sceneIndex)

let device = MTLCreateSystemDefaultDevice()!

var randomNumbers = (0..<1_000_000).map { _ in Float.random(in: 0..<1) }
var hitRecord: [hitRC] = Array(repeating: hitRC(), count: Int(renderSetting.imageWidth * renderSetting.imageHeight))
var atten: [color] = Array(repeating: color(), count: Int(renderSetting.imageWidth * renderSetting.imageHeight))
var scattered: [ray] = Array(repeating: ray(), count: Int(renderSetting.imageWidth * renderSetting.imageHeight))
var Image: [UInt8] = Array(repeating: UInt8(), count: Int(renderSetting.imageWidth * renderSetting.imageHeight) * 3)
var randomVector: [vec] = Array(repeating: vec(), count: 256)
var permuteX: [Int32] = Array (repeating: Int32(), count: 256)
var permuteY: [Int32] = Array (repeating: Int32(), count: 256)
var permuteZ: [Int32] = Array (repeating: Int32(), count: 256)
var size : Int32 = Int32(sceneSet.count)
var debug: [Float] = Array(repeating: Float(), count: 100)

let library = device.makeDefaultLibrary()!
let function = library.makeFunction(name: "render")!
var computePipelineState: MTLComputePipelineState
do {
    computePipelineState = try device.makeComputePipelineState(function: function)
} catch {
    fatalError("无法创建 computePipelineState: \(error)")
}

let commandQueue = device.makeCommandQueue()!
let commandBuffer = commandQueue.makeCommandBuffer()!
let computeEncoder = commandBuffer.makeComputeCommandEncoder()!
computeEncoder.setComputePipelineState(computePipelineState)

var randomNumbersBuffer = device.makeBuffer(bytes: randomNumbers,
                                            length: randomNumbers.count * MemoryLayout<Float>.stride,
                                            options: .storageModeShared)!
var randomIndexBuffer = device.makeBuffer(length: MemoryLayout<UInt32>.stride,
                                          options: .storageModeShared)!
var randomIndex = randomIndexBuffer.contents().bindMemory(to: UInt32.self, capacity: 1)
randomIndex.pointee = 0
var hitRecordBuffer = device.makeBuffer(bytes: hitRecord,
                                        length: hitRecord.count * MemoryLayout<hitRC>.stride,
                                        options: .storageModeShared)!
var sceneSetBuffer = device.makeBuffer(bytes: sceneSet,
                                         length: sceneSet.count * MemoryLayout<scene>.stride,
                                         options: .storageModeShared)!
var attenBuffer = device.makeBuffer(bytes: atten,
                                    length: atten.count * MemoryLayout<color>.stride,
                                    options: .storageModeShared)!
var scatteredBuffer = device.makeBuffer(bytes: scattered,
                                        length: scattered.count * MemoryLayout<ray>.stride,
                                        options: .storageModeShared)!
var renderSettingBuffer = device.makeBuffer(bytes: &renderSetting, length: MemoryLayout<renderSettings>.stride, options: .storageModeShared)!
var ImageBuffer = device.makeBuffer(bytes: Image,
                                         length: Int(renderSetting.imageWidth * renderSetting.imageHeight) * 3 * MemoryLayout<UInt8>.stride,
                                         options: .storageModeShared)!
var imageTextureSetBuffer = device.makeBuffer(bytes: imageTextureSet,
                                        length: imageTextureSet.count * MemoryLayout<UInt8>.stride,
                                        options: .storageModeShared)!
var randomVectorBuffer = device.makeBuffer(bytes: randomVector,
                                        length: 256 * MemoryLayout<vec>.stride,
                                        options: .storageModeShared)!
var permuteXBuffer = device.makeBuffer(bytes: permuteX,
                                        length: 256 * MemoryLayout<Int32>.stride,
                                        options: .storageModeShared)!
var permuteYBuffer = device.makeBuffer(bytes: permuteY,
                                        length: 256 * MemoryLayout<Int32>.stride,
                                        options: .storageModeShared)!
var permuteZBuffer = device.makeBuffer(bytes: permuteZ,
                                        length: 256 * MemoryLayout<Int32>.stride,
                                        options: .storageModeShared)!
var sizeBuffer = device.makeBuffer(bytes: &size, length: MemoryLayout<Int32>.stride, options: .storageModeShared)!
var debugBuffer = device.makeBuffer(bytes: debug,
                                    length: 100 * MemoryLayout<Float>.stride,
                                    options: .storageModeShared)!

computeEncoder.setBuffer(randomIndexBuffer, offset: 0, index: 0)
computeEncoder.setBuffer(randomNumbersBuffer, offset: 0, index: 1)
computeEncoder.setBuffer(hitRecordBuffer, offset: 0, index: 2)
computeEncoder.setBuffer(sceneSetBuffer, offset: 0, index: 3)
computeEncoder.setBuffer(renderSettingBuffer, offset: 0, index: 4)
computeEncoder.setBuffer(attenBuffer, offset: 0, index: 5)
computeEncoder.setBuffer(scatteredBuffer, offset: 0, index: 6)
computeEncoder.setBuffer(randomVectorBuffer, offset: 0, index: 7)
computeEncoder.setBuffer(permuteXBuffer, offset: 0, index: 8)
computeEncoder.setBuffer(permuteYBuffer, offset: 0, index: 9)
computeEncoder.setBuffer(permuteZBuffer, offset: 0, index: 10)
computeEncoder.setBuffer(ImageBuffer, offset: 0, index: 11)
computeEncoder.setBuffer(imageTextureSetBuffer, offset: 0, index: 12)
computeEncoder.setBuffer(sizeBuffer, offset: 0, index: 13)
computeEncoder.setBuffer(debugBuffer, offset: 0, index: 14)

var threadGroupSize = MTLSize(width: 16, height: 16, depth: 1)
var gridSize = MTLSize(width: Int(renderSetting.imageWidth), height: Int(renderSetting.imageHeight), depth: 1)

let startTime = CFAbsoluteTimeGetCurrent()

computeEncoder.dispatchThreads(gridSize, threadsPerThreadgroup: threadGroupSize)
computeEncoder.endEncoding()
commandBuffer.commit()
commandBuffer.waitUntilCompleted()

let endTime = CFAbsoluteTimeGetCurrent()
let gpuTime = endTime - startTime

print ("渲染时间：\(gpuTime)s")

var finalImageData = ImageBuffer.contents()
var resultPointer = finalImageData.assumingMemoryBound(to: UInt8.self)

var debugData = debugBuffer.contents()
var debugPointer = debugData.assumingMemoryBound(to: Float.self)

for i in 0..<5 {
    print ("debug槽\(i): \((debugPointer+i).pointee)")
}

var imageWidth = renderSetting.imageWidth
var imageHeight = renderSetting.imageHeight
var totalPixels = imageWidth * imageHeight
var fileURL = URL(fileURLWithPath: "/Users/LimeEcho/Documents/DieInTheLight/DieInTheLight/output.ppm")

do {
    var outputString = "P3\n\(imageWidth) \(imageHeight)\n255\n"
    
    for i in 0..<totalPixels {
        let r = (resultPointer + Int(i) * 3).pointee
        let g = (resultPointer + Int(i * 3 + 1)).pointee
        let b = (resultPointer + Int(i * 3 + 2)).pointee
        
        outputString += "\(r) \(g) \(b)\n"
    }
    
    try outputString.write(to: fileURL, atomically: true, encoding: .utf8)
    
    print ("数据已成功写入: \(fileURL.path)")
} catch {
    print ("写入文件失败: \(error)")
}

Can you tell me how to write the makefile or just how to compile them and link to a single file?

I don't know Xcode. Is there a way to see the commands it executes when it builds your project? If so, you can just cut and paste those commands as the recipes in a makefile (at least as a first step). Once it works you can go back and clean up the makefile to use make variables etc. — MadScientist
– MadScientist, Commented May 1 at 13:06
Yes, there is.. But they are soooo complex. There are sdk version, compile target, language version etc. It is nearly impossible to just 'copy, paste and run':( @MadScientist — deepinFolk
– deepinFolk, Commented May 1 at 13:46
You seem to imply that Xcode would "link to a single file". Did you confirm this? Because as far as I know, Xcode would have produced an executable binary, and some .metallib files. It does not just produce one single file. Perhaps you want to create an application bundle (.app) to bundle all of these output files together? — Sweeper
– Sweeper, Commented May 1 at 14:11

Sweeper · Accepted Answer · 2025-05-02 07:10:50Z

but I just don't know how to link them together to a single binary file.

There is no single file.

The .metallib files should be separate from the executable binary. You cannot link them together.

The .metallib files are not part of the executable binary in the same way that image files that your program uses are not part of the executable binary. From Swift's POV, they are all just "resources". When you write device.makeDefaultLibrary(), it essentially just finds a default.metallib file in the same directory as the executable and reads that.

So all you need to do is

create the executable binary - let's call this my_game

 swiftc file1.swift file2.swift files.swift -o my_game

create the default.metallib

 xcrun -sdk macosx metal file1.metal file2.metal

Generating object files first

If you want to use make do what Xcode does, compiling each file separately to object files first and then linking them, you can do that too.

To get a single file1.o from a file1.swift that depends on things declared in file2.swift, you can do (this is simplified from Xcode's build output)

swift frontend -module-name SomeModuleName \
-c file2.swift -primary-file file1.swift \
-sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk

Note the -primary-file option is used to specify which file you want to compile. Remember to use the same module name for all the files.

Then you can link the object files with swiftc file1.o file2.o -o my_game.

For Metal, you can compile one single metal file to .air file like this

xcrun -sdk macosx metal -c -frecord-sources file1.metal

You can link .air files into a .metallib like this:

xcrun -sdk macosx metal -frecord-sources -o default.metallib file1.air file2.air file3.air

Putting the files in a bundle

If you put these in the same directory, you will be able to run my_game.

If you want to bundle these into "one thing", you can put them in an .app bundle. This is basically a regular directory except it doesn't look like a directory in Finder. The structure should look like:

My Game.app
└── Contents
    ├── Info.plist
    ├── MacOS
    │   └── my_game // this is the executable file
    └── Resources
        └── default.metallib

Here is a very minimal example of Info.plist. For more documentation see here.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>CFBundleExecutable</key>
    <string>my_game</string>
    <key>CFBundleIdentifier</key>
    <string>com.example.mygame</string>
    <key>CFBundleInfoDictionaryVersion</key>
    <string>6.0</string>
    <key>CFBundleName</key>
    <string>My Game</string>
    <key>CFBundleShortVersionString</key>
    <string>1.0</string>
    <key>CFBundleVersion</key>
    <string>1</string>
</dict>
</plist>

ohhhh my god thank you soo much!!! I completely understood! So I cannot just make them into one executable, instead I have to make two files default.metallib and the executable from Swift! I have found that device.makeDefaultLibrary() do find the default.metallib in the same dictionary. And that is enough 'cause I just want to convert my CPU program to GPU. SwiftUI will be added after my GPU code is fully tested, so I needn't pack them into an .app file. Thank you so much for explaining it to me! ; )

Collectives™ on Stack Overflow

How to convert an Xcode project that uses automatic build to a command-line project using make?

1 Answer 1

There is no single file.

Generating object files first

Putting the files in a bundle

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

There is no single file.

Generating object files first

Putting the files in a bundle

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related