<div>ConsultBae is a people-powered recruitment &amp; IT organization. We help companies build amazing tech products with Staff Augmentation empowered by Technology &amp; Innovation. Our Services Include Digital Transformation, Workforce Hiring Services, IT Staffing, AI &amp; ML Product Solutions to B2B Business.</div>

ConsultBae India Private limited

Job Title: LLM - Engineering Expert
Employment Type: Contract Positions Open: 210 Duration: Up to 24 weeks Start Date: Immediate Location: Remote (Bangladesh, India, Indonesia, Egypt, Ghana, Kenya, Nigeria, Turkey, Brazil, Colombia) Work Hours: Full-time, 40 hrs/week with 4 hours overlap with PST Rate: $300 per task (considering 30-40 hours AHT per task) 
Key Skills: Python, ngspice, PySpice, OpenFOAM, FEniCSx, CalculiX, python-control, CadQuery, build123d, OpenModelica, Cantera, Gmsh, LLM Evaluation, Coding Agents, Failure-Mode Analysis, Trajectory Analysis, Engineering Simulation, Electrical Engineering, Mechanical Engineering, Aerospace Engineering, Control Systems, Robotics
About the Role We are seeking experienced AI Evaluation Engineers (Engineering Simulation &amp; Design) to author and validate "model-breaking," simulation-based engineering design problems to train and evaluate state-of-the-art AI agents. Operating across major engineering disciplines—including Electrical, Mechanical, Control Systems, Aerospace, Systems, and Robotics—you will create complex, multi-constraint tasks where AI agents must interpret requirements, navigate trade-offs, configure open-source simulation tools, diagnose failures, and iterate toward valid solutions. You will analyze agent execution logs, expose systemic reasoning gaps, and build automated, objective graders to elevate frontier model performance.
Job Requirements
<ul>
<li>Education &amp; Expertise: Master's degree or PhD in Electrical, Mechanical, Aerospace, with 10+ years of hands-on engineering design experience.</li>
<li>Simulation Tooling: Proficiency with at least one domain-relevant open-source simulation package (e.g., ngspice, PySpice, OpenFOAM, FEniCSx, CalculiX, python-control, CadQuery, build123d, OpenModelica, Cantera, Gmsh) combined with strong Python scripting skills.</li>
<li>AI Evaluation &amp; Failure Diagnostics: Hands-on experience with modern LLMs/coding agents and evaluation concepts (pass@k, failure-mode analysis, nondeterministic behavior), with the ability to audit trajectory logs and isolate core reasoning/tool-use failures.</li>
<li>Domain Rigor &amp; Precision: Uncompromising attention to physical plausibility, unit consistency, boundary conditions, convergence criteria, and technical documentation.</li>
<li>Availability &amp; Commitment: Talent must have weekend on-call availability (part-time engagement is acceptable).</li>
<li>Technical Infrastructure: Personal desktop/laptop equipped with a stable, high-speed internet connection in a remote setup.</li>
</ul>
Job Responsibilities
<ul>
<li>Model-Breaking Problem Design: Author original, self-contained engineering design tasks with competing constraints, explicit optimization targets, validated reference solutions, and objective autograders.</li>
<li>Environment &amp; Simulation Integration: Build, run, and validate problem environments using open-source simulation tools and custom Python test benches.</li>
<li>Trajectory Analysis &amp; Failure Mode Taxonomy: Evaluate coding agent outputs and execution logs across repeated trials to identify systemic failure modes (e.g., misinterpreting simulator feedback, premature design convergence, physically impossible geometries).</li>
<li>Difficulty Calibration &amp; Benchmark Refinement: Iteratively refine problem difficulty based on empirical model performance data without introducing ambiguity or missing information.</li>
<li>Cross-Functional Collaboration: Partner with AI researchers, pod leads, and domain experts to integrate high-rigor benchmarks into the model evaluation pipeline.</li>
</ul>
Domains
<ul>
<li>Electrical Engineering</li>
<li>Mechanical Engineering</li>
<li>Aerospace Engineering</li>
</ul>
Education &amp; Experience
<ul>
<li>Bachelor's degree or equivalent practical experience in any field.</li>
<li>Experience in AI evaluation, data annotation, content review, quality assurance, or a related analytical role is preferred but not required.</li>
</ul>
Offer Details
<ul>
<li>Commitment: 40 hours/week with 4 hours overlap with PST</li>
<li>Employment Type: Contractor</li>
<li>Duration: Up to 24 weeks</li>
</ul>