I have recently been working on an AI compiler project that requires a GPU. I do have an old GTX 1650, but recent versions of nvcc have dropped support for machines with sm_70 and sm_75. That left me with the options of renting a GPU from vast.ai or AutoDL, but both platforms have their own problems:

AutoDL: it uses containers, which do not support using Nsight Compute to collect hardware performance counters.

vast.ai: its virtual machines can use Nsight Compute, but pricing is in US dollars, so I would not want to keep one running for too long.

I knew the university had GPUs, although everyone in CSE says that there are never enough of them—which is indeed true. UNSW’s Katana HPC system has more than a hundred GPUs and is available to HDR students. I submitted a support ticket a while ago, and it was approved without any trouble. Today I finally gave it a try.

image-20260914231500604

Machine Details

Here is what the official website lists:

  • 35 GPU nodes
    • 80 x Nvidia H200 141GB (13 nodes)
    • 1 x Nvidia H100 94GB (1 node)
    • 1 x Nvidia GH200 480GB + 96GB (1 node)
    • 32 x Nvidia L40S 48GB (7 nodes)
    • 24 x Nvidia A100 40GB (3 nodes)
    • 32 x Nvidia V100 32GB (8 nodes)
    • 16 x Nvidia RTX Pro 6000 96GB (2 nodes)
  • 20 GPU nodes have priority access for the school or group that purchased them
  • 15 GPU nodes are available to all researchers

Well, more than a hundred GPUs is not actually that many. Once they are divided among the entire university, it is only slightly better than having none 🧐

The accompanying examples are very detailed, and Katana appears to use PBS for scheduling: Running Jobs on Katana

Containers should also be usable. Although the manual does not mention them, I do have permission to access Apptainer. See the Apptainer manual for details: Introduction to Apptainer

I originally planned to run tests with Nsight Compute and OpenAI Triton, but the wait for a GPU was simply too long. I will update this post when I have time. Without specifying a GPU, the job should run on a V100. Update: both tests are now complete.

Storage

The default location provides 15 GB, along with an additional 128.0 GB scratch disk:

                                                        Own      Own      Own
                              Own      Own      Own    Inode    Inode    Inode
                             Usage    Quota   % Used   Usage    Quota   % Used
            /home/z1234123    9.3GB   15.0GB     62%     92K      --       --
     /srv/scratch/z1234123      0KB  128.0GB      0%      1      500K       0%

Nsight Compute (A Failure)

Nsight Compute finished running, but the conclusion is that it is completely unusable here. I still need vast.ai after all.

Nsight Compute also cannot use the 2023 version. The test below uses the 2024 version:

z1234123@katana2:~/ncu-test $ cat myjob.pbs.o9316631
Tue Sep 15 08:04:20 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.173.02             Driver Version: 580.173.02     CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  Tesla V100-SXM2-32GB           On  |   00000000:AF:00.0 Off |                    0 |
| N/A   33C    P0             41W /  300W |       0MiB /  32768MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+


+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
 WARN cache for Repodata at /home/z1234123/.cache/rattler/cache/repodata is on a network/parallel filesystem (NFS/SMB/FUSE/BeeGFS/Lustre/GPFS/CephFS), redirected to /scratch/pbs.9316631.kman.restech.unsw.edu.au/pixi-cache-z1234123/repodata for this run. Set [cache.repodata] in config.toml or PIXI_CACHE_DIR to override, or [cache.netfs-redirect] = "never" to keep the original path.
NVIDIA (R) Nsight Compute Command Line Profiler
Copyright (c) 2018-2024 NVIDIA Corporation
Version 2024.3.2.0 (build 34861637) (public-release)
 WARN cache for Repodata at /home/z1234123/.cache/rattler/cache/repodata is on a network/parallel filesystem (NFS/SMB/FUSE/BeeGFS/Lustre/GPFS/CephFS), redirected to /scratch/pbs.9316631.kman.restech.unsw.edu.au/pixi-cache-z1234123/repodata for this run. Set [cache.repodata] in config.toml or PIXI_CACHE_DIR to override, or [cache.netfs-redirect] = "never" to keep the original path.
==PROF== Connected to process 1172731 (/home/z1234123/ncu-test/ncu_test)
==ERROR== ERR_NVGPUCTRPERM - The user does not have permission to access NVIDIA GPU Performance Counters on the target device 0. For instructions on enabling permissions and to get more information see https://developer.nvidia.com/ERR_NVGPUCTRPERM
Allocating 64.00 MB per array
Result sample: 3.000298
==PROF== Disconnected from process 1172731


================================================================================
                     Resource Usage on 15/09/2026 08:04:26


Job Id: 9316631
Queue: CSE
Walltime: 00:00:03  (requested 12:00:00)
Job unsuccessful. Exit Status 1 - Exit status refers to the exit value of the
top process in the job, typically the shell.
This may be the exit value of the last command executed in the shell.
See job output error.


--------------------------------------------------------------------------------
|              |             GPUs              |            Memory             |
--------------------------------------------------------------------------------
| Node  GPU ID | Requested   Used   Efficiency | Available   Used              |
| k106    3    |     1       0.04       4%     | 32768 MiB   0.0B              |
--------------------------------------------------------------------------------
|Total         |     1       0.04      4.0%    |                               |
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------
|              |             CPUs              |            Memory             |
--------------------------------------------------------------------------------
| Node         | Requested   Used   Efficiency | Requested   Used   Efficiency |
| k106         |     1       0.67      67.0%   |   4.0gb    0.42gb     10.5%   |
--------------------------------------------------------------------------------

Running OpenAI Triton

OpenAI Triton currently only runs on sm_80 and newer. According to the AI, support for Volta and Turing was removed starting with version 3.3. A specific GPU must therefore be requested:

pbsnodes -av | grep gpu_model
qsub -I -l select=1:ncpus=2:mem=8gb:ngpus=1:gpu_model=A100

Getting an A100 allocation is still somewhat difficult, with an average wait of more than six hours. Once allocated, however, it runs without any problems.

I used uv to configure the environment. The following is a working pyproject.toml:

[project]
name = "triton-test"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.6"
dependencies = [
    "numpy>=1.19.5",
    "torch>=1.5.1",
    "triton>=3.1.0",
]


[project.scripts]
triton-test = "triton_test:main"


[build-system]
requires = ["uv_build>=0.12.13,<0.13.0"]
build-backend = "uv_build"

Initialize the environment as follows. The first run may also require uv python install 3.12:

uv init
uv venv --python 3.12
uv sync

Test code:

import torch
import triton
import triton.language as tl




@triton.jit
def vector_add_kernel(
    x_ptr,
    y_ptr,
    out_ptr,
    n_elements: tl.constexpr,
    BLOCK_SIZE: tl.constexpr,
):
    pid = tl.program_id(axis=0)


    offsets = pid * BLOCK_SIZE + tl.arange(0, BLOCK_SIZE)
    mask = offsets < n_elements


    x = tl.load(x_ptr + offsets, mask=mask)
    y = tl.load(y_ptr + offsets, mask=mask)


    tl.store(
        out_ptr + offsets,
        x + y,
        mask=mask,
    )




def main():
    print("PyTorch version :", torch.__version__)
    print("Triton version  :", triton.__version__)
    print("CUDA available  :", torch.cuda.is_available())


    if not torch.cuda.is_available():
        raise RuntimeError("CUDA is not available")


    device = torch.cuda.current_device()
    props = torch.cuda.get_device_properties(device)


    print("GPU             :", props.name)
    print(
        "Compute capability:",
        f"{props.major}.{props.minor}",
    )


    n = 10_000_003


    x = torch.randn(
        n,
        device="cuda",
        dtype=torch.float32,
    )


    y = torch.randn(
        n,
        device="cuda",
        dtype=torch.float32,
    )


    output = torch.empty_like(x)


    BLOCK_SIZE = 1024


    grid = (
        triton.cdiv(n, BLOCK_SIZE),
    )


    vector_add_kernel[grid](
        x,
        y,
        output,
        n,
        BLOCK_SIZE=BLOCK_SIZE,
    )


    torch.cuda.synchronize()


    expected = x + y


    max_error = (
        output - expected
    ).abs().max().item()


    correct = torch.allclose(
        output,
        expected,
        rtol=1e-5,
        atol=1e-6,
    )


    print("Max error       :", max_error)
    print("Result correct  :", correct)


    if not correct:
        raise RuntimeError(
            "Triton kernel produced incorrect result"
        )


    print("\nTriton works correctly on this GPU.")




if __name__ == "__main__":
    main()

PBS file used to submit the test:

#!/bin/bash


#PBS -l select=1:ncpus=1:mem=4gb:ngpus=1:gpu_model=A100
#PBS -l walltime=12:00:00
#PBS -M z1234123@ad.unsw.edu.au
#PBS -m ae
#PBS -j oe




cd $PBS_O_WORKDIR


uv run python test_triton_a100.py

Test results:

z1234123@katana2:~/triton-test $ cat triton.pbs.o9317745
PyTorch version : 2.14.0+cu130
Triton version  : 3.8.0
CUDA available  : True
GPU             : NVIDIA A100-PCIE-40GB
Compute capability: 8.0
Max error       : 0.0
Result correct  : True


Triton works correctly on this GPU.


================================================================================
                     Resource Usage on 15/09/2026 19:27:40


Job Id: 9317745
Queue: CSE
Walltime: 00:00:09  (requested 12:00:00)
Job execution was successful. Exit Status 0.


--------------------------------------------------------------------------------
|              |             GPUs              |            Memory             |
--------------------------------------------------------------------------------
| Node  GPU ID | Requested   Used   Efficiency | Available   Used              |
| k092    2    |     1       0.01       1%     | 40960 MiB  814.0M             |
--------------------------------------------------------------------------------
|Total         |     1       0.01      1.0%    |                               |
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------
|              |             CPUs              |            Memory             |
--------------------------------------------------------------------------------
| Node         | Requested   Used   Efficiency | Requested   Used   Efficiency |
| k092         |     1       0.67      67.0%   |   4.0gb     2.1gb     52.5%   |
--------------------------------------------------------------------------------

It is fine for some free experimentation, but if you actually want to run benchmarks, you will probably spend most of your time waiting.

Running TileLang

I thought of TileLang because it still supports sm_70 and sm_75.

uv init
uv venv --python 3.12
uv pip install --no-cache torch==2.14.0 --index-url https://download.pytorch.org/whl/cu126
uv pip install --no-cache tilelang

Test code:

import torch
import tilelang
import tilelang.language as T


print("=" * 60)
print("Environment")
print("=" * 60)


print("TileLang version:", tilelang.__version__)
print("PyTorch version :", torch.__version__)
print("CUDA available  :", torch.cuda.is_available())


if not torch.cuda.is_available():
    raise RuntimeError("CUDA GPU is not visible inside PBS job")


print("GPU name        :", torch.cuda.get_device_name(0))
print("CUDA capability :", torch.cuda.get_device_capability(0))


major, minor = torch.cuda.get_device_capability(0)


if (major, minor) != (7, 0):
    print(
        f"WARNING: expected V100 / sm_70, "
        f"but got sm_{major}{minor}"
    )


print("=" * 60)
print("TileLang vector-add test")
print("=" * 60)




@tilelang.jit(target={"kind": "cuda", "arch": "sm_70"})
def vector_add(N: int, block: int = 256):


    @T.prim_func
    def kernel(
        A: T.Tensor((N,), "float32"),
        B: T.Tensor((N,), "float32"),
        C: T.Tensor((N,), "float32"),
    ):
        with T.Kernel(T.ceildiv(N, block), threads=block) as bx:
            for tx in T.Parallel(block):
                idx = bx * block + tx
                C[idx] = A[idx] + B[idx]


    return kernel




N = 1 << 20


a = torch.randn(N, device="cuda", dtype=torch.float32)
b = torch.randn(N, device="cuda", dtype=torch.float32)
c = torch.empty_like(a)


print("Compiling TileLang kernel for sm_70 ...")


kernel = vector_add(N)


torch.cuda.synchronize()


print("Running kernel ...")


kernel(a, b, c)


torch.cuda.synchronize()


ref = a + b


torch.testing.assert_close(c, ref)


max_error = (c - ref).abs().max().item()


print("Max error:", max_error)
print()
print("PASS: TileLang successfully compiled and ran on V100 / sm_70.")

Working PBS file:

#!/bin/bash


#PBS -l select=1:ncpus=1:mem=4gb:ngpus=1:gpu_model=V100
#PBS -l walltime=01:00:00
#PBS -M z1234123@ad.unsw.edu.au
#PBS -m ae
#PBS -j oe




cd $PBS_O_WORKDIR


module load gcc/12.2.0
module load cuda/12.8


uv run python test_tilelang_v100.py

Test results:

============================================================
Environment
============================================================
TileLang version: 0.1.14
PyTorch version : 2.14.0+cu126
CUDA available  : True
GPU name        : Tesla V100-SXM2-32GB
CUDA capability : (7, 0)
============================================================
TileLang vector-add test
============================================================
Compiling TileLang kernel for sm_70 ...
2026-09-15 22:10:27  [TileLang:tilelang.jit.kernel:INFO] (kernel.py:137): TileLang begins to compile kernel `kernel` with `out_idx=None`
2026-09-15 22:10:29  [TileLang:tilelang.jit.kernel:INFO] (kernel.py:145): TileLang completes to compile kernel `kernel`
Running kernel ...
Max error: 0.0


PASS: TileLang successfully compiled and ran on V100 / sm_70.


================================================================================
                     Resource Usage on 15/09/2026 22:10:46


Job Id: 9319953
Queue: CSE
Walltime: 00:00:26  (requested 01:00:00)
Job execution was successful. Exit Status 0.


--------------------------------------------------------------------------------
|              |             GPUs              |            Memory             |
--------------------------------------------------------------------------------
| Node  GPU ID | Requested   Used   Efficiency | Available   Used              |
| k105    1    |     1        0.0       0%     | 32768 MiB  410.0M             |
--------------------------------------------------------------------------------
|Total         |     1        0.0      0.0%    |                               |
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------
|              |             CPUs              |            Memory             |
--------------------------------------------------------------------------------
| Node         | Requested   Used   Efficiency | Requested   Used   Efficiency |
| k105         |     1       0.31      31.0%   |   4.0gb     0.7gb     17.5%   |
--------------------------------------------------------------------------------

Notes on Using uv

The cache used by uv may fill up the 15 GB of small storage. It is probably better to use conda or a Python virtual environment directly, or enable --no-cache when using uv.

Tips

qstat -u "$USER"

This shows all jobs you have submitted.

module avail gcc

This lists the available GCC versions. TileLang appears to use CuTe templates, and GCC versions older than 10 may not work.

Conclusion

I had originally planned to try the CSE HPC system as well, but CSE gives each person only 3 GB of shared storage for teaching use. A single Conda environment can fill that up, so I can only hope that they provide more storage in the future.