LLM Pre-Training Engineer

E-Group is an innovative deep-tech, high-tech mid-market company with a strong vision, a fast-growing and friendly atmosphere, and a dynamic team of experts working on next-generation, beyond-state-of-the-art, future-proof technologies in artificial intelligence core technology stack, payment transactions, and digital identification.


Since its foundation in 1993, E-Group has gained a strong track record of successful projects in the healthcare, financial, energy, and government sectors. As an active member of the European innovation ecosystem, it leads or participates in many top-tier projects each year with EU Institutions, European universities, companies, and financial institutions to apply the latest scientific advances to real customer needs.

We are looking for YOU if:

  • The “impossible” for you is just another challenge
  • You inspire your environment with your original way of thinking
  • You want to be involved in world-class, professionally inspiring development projects

Main tasks:

  • Design and implement tensor, pipeline, MoE and ZeRO-3 parallelism strategies for LLM models
  • Build real-time utilisation dashboards and autoscale GPU clusters (Slurm / Karpenter)
  • Implement fault-tolerant checkpointing and rapid resume logic for spot-instance fleets

The ideal candidate can:

  • Forecast GPU-hour budgets and advise on reserved vs spot capacity strategies
  • Run continuous evaluation sweeps (MMLU, HELM) and interpret scaling-law curves
  • Work together with DevOps on cluster events and with ML engineers on model specs

What we expect:

  • Deep expertise in PyTorch Distributed, FSDP, DeepSpeed or Megatron-Core
  • Demonstrated success training multi-billion-parameter models at scale
  • Familiarity with data-engineering stacks (Ray Data, Petastorm, WebDataset)
  • Solid CUDA background and ability to write/modify custom GPU kernels when needed

What We Offer:

  • Competitive salary commensurate with experience
  • Real professional challenges where you can apply and develop your skills
  • A stable corporate environment
  • Long-term job opportunities
  • Varied international projects
  • Modern, technological environment, modern equipment
  • Friendly, supportive, and professionally committed team
  • Personal and professional development opportunities
  • A pleasant working environment in Buda, surrounded by greenery, in a family atmosphere

If you know yourself from the description, then E-Group is the place for you.

We care about who you are, your goals, and what person you want to become. We offer a place where you can add value not only with your skills, but also with your personality.

We look forward to receiving your application with a photo CV at hr@egroup.hu.

Share this job
This site is registered on wpml.org as a development site.