Skip to content

Latest commit

 

History

History
678 lines (324 loc) · 27.1 KB

File metadata and controls

678 lines (324 loc) · 27.1 KB

LLM Features, Prompting Patterns & Structured Calls

Skills #4409–#4694 (286 techniques). Part of Awesome Token Saving.

Support the project: https://saweria.co/0xgetz


LLM feature: function calling

4409. Use the leaner path — prefer function calling when it returns less text than free-form generation.

4410. Constrain the output — pair function calling with a tight schema to avoid prose.

4411. Cache {C} results — memoize deterministic function calling outputs keyed on inputs.

4412. Right-size inputs to {C} — trim inputs before function calling so it processes less.

4413. Cap tokens with {C} — set output limits when using function calling.

4414. Batch with {C} — combine many items through function calling to share overhead.

4415. Skip {C} when rules suffice — handle obvious cases in code before invoking function calling.

4416. Measure {C} cost — instrument function calling calls to find expensive usage.

LLM feature: structured outputs

4417. Use the leaner path — prefer structured outputs when it returns less text than free-form generation.

4418. Constrain the output — pair structured outputs with a tight schema to avoid prose.

4419. Cache {C} results — memoize deterministic structured outputs outputs keyed on inputs.

4420. Right-size inputs to {C} — trim inputs before structured outputs so it processes less.

4421. Cap tokens with {C} — set output limits when using structured outputs.

4422. Batch with {C} — combine many items through structured outputs to share overhead.

4423. Skip {C} when rules suffice — handle obvious cases in code before invoking structured outputs.

4424. Measure {C} cost — instrument structured outputs calls to find expensive usage.

LLM feature: vision input

4425. Use the leaner path — prefer vision input when it returns less text than free-form generation.

4426. Constrain the output — pair vision input with a tight schema to avoid prose.

4427. Cache {C} results — memoize deterministic vision input outputs keyed on inputs.

4428. Right-size inputs to {C} — trim inputs before vision input so it processes less.

4429. Cap tokens with {C} — set output limits when using vision input.

4430. Batch with {C} — combine many items through vision input to share overhead.

4431. Skip {C} when rules suffice — handle obvious cases in code before invoking vision input.

4432. Measure {C} cost — instrument vision input calls to find expensive usage.

LLM feature: audio input

4433. Use the leaner path — prefer audio input when it returns less text than free-form generation.

4434. Constrain the output — pair audio input with a tight schema to avoid prose.

4435. Cache {C} results — memoize deterministic audio input outputs keyed on inputs.

4436. Right-size inputs to {C} — trim inputs before audio input so it processes less.

4437. Cap tokens with {C} — set output limits when using audio input.

4438. Batch with {C} — combine many items through audio input to share overhead.

4439. Skip {C} when rules suffice — handle obvious cases in code before invoking audio input.

4440. Measure {C} cost — instrument audio input calls to find expensive usage.

LLM feature: batch API

4441. Use the leaner path — prefer batch API when it returns less text than free-form generation.

4442. Constrain the output — pair batch API with a tight schema to avoid prose.

4443. Cache {C} results — memoize deterministic batch API outputs keyed on inputs.

4444. Right-size inputs to {C} — trim inputs before batch API so it processes less.

4445. Cap tokens with {C} — set output limits when using batch API.

4446. Batch with {C} — combine many items through batch API to share overhead.

4447. Skip {C} when rules suffice — handle obvious cases in code before invoking batch API.

4448. Measure {C} cost — instrument batch API calls to find expensive usage.

LLM feature: moderation endpoint

4449. Use the leaner path — prefer moderation endpoint when it returns less text than free-form generation.

4450. Constrain the output — pair moderation endpoint with a tight schema to avoid prose.

4451. Cache {C} results — memoize deterministic moderation endpoint outputs keyed on inputs.

4452. Right-size inputs to {C} — trim inputs before moderation endpoint so it processes less.

4453. Cap tokens with {C} — set output limits when using moderation endpoint.

4454. Batch with {C} — combine many items through moderation endpoint to share overhead.

4455. Skip {C} when rules suffice — handle obvious cases in code before invoking moderation endpoint.

4456. Measure {C} cost — instrument moderation endpoint calls to find expensive usage.

LLM feature: embeddings endpoint

4457. Use the leaner path — prefer embeddings endpoint when it returns less text than free-form generation.

4458. Constrain the output — pair embeddings endpoint with a tight schema to avoid prose.

4459. Cache {C} results — memoize deterministic embeddings endpoint outputs keyed on inputs.

4460. Right-size inputs to {C} — trim inputs before embeddings endpoint so it processes less.

4461. Cap tokens with {C} — set output limits when using embeddings endpoint.

4462. Batch with {C} — combine many items through embeddings endpoint to share overhead.

4463. Skip {C} when rules suffice — handle obvious cases in code before invoking embeddings endpoint.

4464. Measure {C} cost — instrument embeddings endpoint calls to find expensive usage.

LLM feature: fine-tuning

4465. Use the leaner path — prefer fine-tuning when it returns less text than free-form generation.

4466. Constrain the output — pair fine-tuning with a tight schema to avoid prose.

4467. Cache {C} results — memoize deterministic fine-tuning outputs keyed on inputs.

4468. Right-size inputs to {C} — trim inputs before fine-tuning so it processes less.

4469. Cap tokens with {C} — set output limits when using fine-tuning.

4470. Batch with {C} — combine many items through fine-tuning to share overhead.

4471. Skip {C} when rules suffice — handle obvious cases in code before invoking fine-tuning.

4472. Measure {C} cost — instrument fine-tuning calls to find expensive usage.

LLM feature: assistants API

4473. Use the leaner path — prefer assistants API when it returns less text than free-form generation.

4474. Constrain the output — pair assistants API with a tight schema to avoid prose.

4475. Cache {C} results — memoize deterministic assistants API outputs keyed on inputs.

4476. Right-size inputs to {C} — trim inputs before assistants API so it processes less.

4477. Cap tokens with {C} — set output limits when using assistants API.

4478. Batch with {C} — combine many items through assistants API to share overhead.

4479. Skip {C} when rules suffice — handle obvious cases in code before invoking assistants API.

4480. Measure {C} cost — instrument assistants API calls to find expensive usage.

LLM feature: realtime API

4481. Use the leaner path — prefer realtime API when it returns less text than free-form generation.

4482. Constrain the output — pair realtime API with a tight schema to avoid prose.

4483. Cache {C} results — memoize deterministic realtime API outputs keyed on inputs.

4484. Right-size inputs to {C} — trim inputs before realtime API so it processes less.

4485. Cap tokens with {C} — set output limits when using realtime API.

4486. Batch with {C} — combine many items through realtime API to share overhead.

4487. Skip {C} when rules suffice — handle obvious cases in code before invoking realtime API.

4488. Measure {C} cost — instrument realtime API calls to find expensive usage.

LLM feature: code interpreter

4489. Use the leaner path — prefer code interpreter when it returns less text than free-form generation.

4490. Constrain the output — pair code interpreter with a tight schema to avoid prose.

4491. Cache {C} results — memoize deterministic code interpreter outputs keyed on inputs.

4492. Right-size inputs to {C} — trim inputs before code interpreter so it processes less.

4493. Cap tokens with {C} — set output limits when using code interpreter.

4494. Batch with {C} — combine many items through code interpreter to share overhead.

4495. Skip {C} when rules suffice — handle obvious cases in code before invoking code interpreter.

4496. Measure {C} cost — instrument code interpreter calls to find expensive usage.

LLM feature: file search

4497. Use the leaner path — prefer file search when it returns less text than free-form generation.

4498. Constrain the output — pair file search with a tight schema to avoid prose.

4499. Cache {C} results — memoize deterministic file search outputs keyed on inputs.

4500. Right-size inputs to {C} — trim inputs before file search so it processes less.

4501. Cap tokens with {C} — set output limits when using file search.

4502. Batch with {C} — combine many items through file search to share overhead.

4503. Skip {C} when rules suffice — handle obvious cases in code before invoking file search.

4504. Measure {C} cost — instrument file search calls to find expensive usage.

LLM feature: JSON mode

4505. Use the leaner path — prefer JSON mode when it returns less text than free-form generation.

4506. Constrain the output — pair JSON mode with a tight schema to avoid prose.

4507. Cache {C} results — memoize deterministic JSON mode outputs keyed on inputs.

4508. Right-size inputs to {C} — trim inputs before JSON mode so it processes less.

4509. Cap tokens with {C} — set output limits when using JSON mode.

4510. Batch with {C} — combine many items through JSON mode to share overhead.

4511. Skip {C} when rules suffice — handle obvious cases in code before invoking JSON mode.

4512. Measure {C} cost — instrument JSON mode calls to find expensive usage.

LLM feature: tool use

4513. Use the leaner path — prefer tool use when it returns less text than free-form generation.

4514. Constrain the output — pair tool use with a tight schema to avoid prose.

4515. Cache {C} results — memoize deterministic tool use outputs keyed on inputs.

4516. Right-size inputs to {C} — trim inputs before tool use so it processes less.

4517. Cap tokens with {C} — set output limits when using tool use.

4518. Batch with {C} — combine many items through tool use to share overhead.

4519. Skip {C} when rules suffice — handle obvious cases in code before invoking tool use.

4520. Measure {C} cost — instrument tool use calls to find expensive usage.

LLM feature: logprobs

4521. Use the leaner path — prefer logprobs when it returns less text than free-form generation.

4522. Constrain the output — pair logprobs with a tight schema to avoid prose.

4523. Cache {C} results — memoize deterministic logprobs outputs keyed on inputs.

4524. Right-size inputs to {C} — trim inputs before logprobs so it processes less.

4525. Cap tokens with {C} — set output limits when using logprobs.

4526. Batch with {C} — combine many items through logprobs to share overhead.

4527. Skip {C} when rules suffice — handle obvious cases in code before invoking logprobs.

4528. Measure {C} cost — instrument logprobs calls to find expensive usage.

LLM feature: system fingerprint

4529. Use the leaner path — prefer system fingerprint when it returns less text than free-form generation.

4530. Constrain the output — pair system fingerprint with a tight schema to avoid prose.

4531. Cache {C} results — memoize deterministic system fingerprint outputs keyed on inputs.

4532. Right-size inputs to {C} — trim inputs before system fingerprint so it processes less.

4533. Cap tokens with {C} — set output limits when using system fingerprint.

4534. Batch with {C} — combine many items through system fingerprint to share overhead.

4535. Skip {C} when rules suffice — handle obvious cases in code before invoking system fingerprint.

4536. Measure {C} cost — instrument system fingerprint calls to find expensive usage.

Pattern: zero-shot

4537. Use only when it pays off — apply zero-shot only where it improves results enough to justify tokens.

4538. Cap the reasoning budget — bound zero-shot so it can't generate runaway thinking tokens.

4539. Hide scratchpad from output — keep zero-shot reasoning internal; return just the answer.

4540. Reduce examples in {C} — trim few-shot/examples used by zero-shot to the minimum.

4541. Cache {C} prefixes — reuse stable zero-shot instruction prefixes across calls.

4542. Summarize between steps — compress intermediate zero-shot steps before the next.

4543. Stop early in {C} — halt zero-shot once a confident answer appears.

4544. Distill {C} into one pass — replace multi-step zero-shot with a single cheaper pass when viable.

Pattern: few-shot

4545. Use only when it pays off — apply few-shot only where it improves results enough to justify tokens.

4546. Cap the reasoning budget — bound few-shot so it can't generate runaway thinking tokens.

4547. Hide scratchpad from output — keep few-shot reasoning internal; return just the answer.

4548. Reduce examples in {C} — trim few-shot/examples used by few-shot to the minimum.

4549. Cache {C} prefixes — reuse stable few-shot instruction prefixes across calls.

4550. Summarize between steps — compress intermediate few-shot steps before the next.

4551. Stop early in {C} — halt few-shot once a confident answer appears.

4552. Distill {C} into one pass — replace multi-step few-shot with a single cheaper pass when viable.

Pattern: chain-of-thought

4553. Use only when it pays off — apply chain-of-thought only where it improves results enough to justify tokens.

4554. Cap the reasoning budget — bound chain-of-thought so it can't generate runaway thinking tokens.

4555. Hide scratchpad from output — keep chain-of-thought reasoning internal; return just the answer.

4556. Reduce examples in {C} — trim few-shot/examples used by chain-of-thought to the minimum.

4557. Cache {C} prefixes — reuse stable chain-of-thought instruction prefixes across calls.

4558. Summarize between steps — compress intermediate chain-of-thought steps before the next.

4559. Stop early in {C} — halt chain-of-thought once a confident answer appears.

4560. Distill {C} into one pass — replace multi-step chain-of-thought with a single cheaper pass when viable.

Pattern: ReAct

4561. Use only when it pays off — apply ReAct only where it improves results enough to justify tokens.

4562. Cap the reasoning budget — bound ReAct so it can't generate runaway thinking tokens.

4563. Hide scratchpad from output — keep ReAct reasoning internal; return just the answer.

4564. Reduce examples in {C} — trim few-shot/examples used by ReAct to the minimum.

4565. Cache {C} prefixes — reuse stable ReAct instruction prefixes across calls.

4566. Summarize between steps — compress intermediate ReAct steps before the next.

4567. Stop early in {C} — halt ReAct once a confident answer appears.

4568. Distill {C} into one pass — replace multi-step ReAct with a single cheaper pass when viable.

Pattern: tree-of-thoughts

4569. Use only when it pays off — apply tree-of-thoughts only where it improves results enough to justify tokens.

4570. Cap the reasoning budget — bound tree-of-thoughts so it can't generate runaway thinking tokens.

4571. Hide scratchpad from output — keep tree-of-thoughts reasoning internal; return just the answer.

4572. Reduce examples in {C} — trim few-shot/examples used by tree-of-thoughts to the minimum.

4573. Cache {C} prefixes — reuse stable tree-of-thoughts instruction prefixes across calls.

4574. Summarize between steps — compress intermediate tree-of-thoughts steps before the next.

4575. Stop early in {C} — halt tree-of-thoughts once a confident answer appears.

4576. Distill {C} into one pass — replace multi-step tree-of-thoughts with a single cheaper pass when viable.

Pattern: self-consistency

4577. Use only when it pays off — apply self-consistency only where it improves results enough to justify tokens.

4578. Cap the reasoning budget — bound self-consistency so it can't generate runaway thinking tokens.

4579. Hide scratchpad from output — keep self-consistency reasoning internal; return just the answer.

4580. Reduce examples in {C} — trim few-shot/examples used by self-consistency to the minimum.

4581. Cache {C} prefixes — reuse stable self-consistency instruction prefixes across calls.

4582. Summarize between steps — compress intermediate self-consistency steps before the next.

4583. Stop early in {C} — halt self-consistency once a confident answer appears.

4584. Distill {C} into one pass — replace multi-step self-consistency with a single cheaper pass when viable.

Pattern: least-to-most

4585. Use only when it pays off — apply least-to-most only where it improves results enough to justify tokens.

4586. Cap the reasoning budget — bound least-to-most so it can't generate runaway thinking tokens.

4587. Hide scratchpad from output — keep least-to-most reasoning internal; return just the answer.

4588. Reduce examples in {C} — trim few-shot/examples used by least-to-most to the minimum.

4589. Cache {C} prefixes — reuse stable least-to-most instruction prefixes across calls.

4590. Summarize between steps — compress intermediate least-to-most steps before the next.

4591. Stop early in {C} — halt least-to-most once a confident answer appears.

4592. Distill {C} into one pass — replace multi-step least-to-most with a single cheaper pass when viable.

Pattern: self-ask

4593. Use only when it pays off — apply self-ask only where it improves results enough to justify tokens.

4594. Cap the reasoning budget — bound self-ask so it can't generate runaway thinking tokens.

4595. Hide scratchpad from output — keep self-ask reasoning internal; return just the answer.

4596. Reduce examples in {C} — trim few-shot/examples used by self-ask to the minimum.

4597. Cache {C} prefixes — reuse stable self-ask instruction prefixes across calls.

4598. Summarize between steps — compress intermediate self-ask steps before the next.

4599. Stop early in {C} — halt self-ask once a confident answer appears.

4600. Distill {C} into one pass — replace multi-step self-ask with a single cheaper pass when viable.

Pattern: reflexion

4601. Use only when it pays off — apply reflexion only where it improves results enough to justify tokens.

4602. Cap the reasoning budget — bound reflexion so it can't generate runaway thinking tokens.

4603. Hide scratchpad from output — keep reflexion reasoning internal; return just the answer.

4604. Reduce examples in {C} — trim few-shot/examples used by reflexion to the minimum.

4605. Cache {C} prefixes — reuse stable reflexion instruction prefixes across calls.

4606. Summarize between steps — compress intermediate reflexion steps before the next.

4607. Stop early in {C} — halt reflexion once a confident answer appears.

4608. Distill {C} into one pass — replace multi-step reflexion with a single cheaper pass when viable.

Pattern: plan-and-solve

4609. Use only when it pays off — apply plan-and-solve only where it improves results enough to justify tokens.

4610. Cap the reasoning budget — bound plan-and-solve so it can't generate runaway thinking tokens.

4611. Hide scratchpad from output — keep plan-and-solve reasoning internal; return just the answer.

4612. Reduce examples in {C} — trim few-shot/examples used by plan-and-solve to the minimum.

4613. Cache {C} prefixes — reuse stable plan-and-solve instruction prefixes across calls.

4614. Summarize between steps — compress intermediate plan-and-solve steps before the next.

4615. Stop early in {C} — halt plan-and-solve once a confident answer appears.

4616. Distill {C} into one pass — replace multi-step plan-and-solve with a single cheaper pass when viable.

Pattern: skeleton-of-thought

4617. Use only when it pays off — apply skeleton-of-thought only where it improves results enough to justify tokens.

4618. Cap the reasoning budget — bound skeleton-of-thought so it can't generate runaway thinking tokens.

4619. Hide scratchpad from output — keep skeleton-of-thought reasoning internal; return just the answer.

4620. Reduce examples in {C} — trim few-shot/examples used by skeleton-of-thought to the minimum.

4621. Cache {C} prefixes — reuse stable skeleton-of-thought instruction prefixes across calls.

4622. Summarize between steps — compress intermediate skeleton-of-thought steps before the next.

4623. Stop early in {C} — halt skeleton-of-thought once a confident answer appears.

4624. Distill {C} into one pass — replace multi-step skeleton-of-thought with a single cheaper pass when viable.

Pattern: RAG prompting

4625. Use only when it pays off — apply RAG prompting only where it improves results enough to justify tokens.

4626. Cap the reasoning budget — bound RAG prompting so it can't generate runaway thinking tokens.

4627. Hide scratchpad from output — keep RAG prompting reasoning internal; return just the answer.

4628. Reduce examples in {C} — trim few-shot/examples used by RAG prompting to the minimum.

4629. Cache {C} prefixes — reuse stable RAG prompting instruction prefixes across calls.

4630. Summarize between steps — compress intermediate RAG prompting steps before the next.

4631. Stop early in {C} — halt RAG prompting once a confident answer appears.

4632. Distill {C} into one pass — replace multi-step RAG prompting with a single cheaper pass when viable.

Function calling specifics

4633. Few tools in scope — Expose only the functions this step needs.

4634. Short tool descriptions — Keep function docs terse but clear.

4635. Enum parameters — Constrain args to enums to shrink schema and output.

4636. Required-only params — Drop optional params you never use.

4637. Compact arg names — Use short, clear parameter names.

4638. Return minimal results — Have functions return only needed fields.

4639. Avoid restating tools — Define functions once, not in messages.

4640. Strict schemas — Use strict mode to prevent extra tokens.

4641. Batch function args — Pass arrays to one call instead of many calls.

4642. Server-side filtering — Filter in the function, not via the model.

4643. IDs over objects — Return IDs the model can resolve later.

4644. Null-free returns — Omit null/empty fields from results.

4645. Cache tool results — Memoize deterministic function outputs.

4646. Short-circuit on cache — Skip the model when a function answers from cache.

4647. Paginate function output — Return small pages, not huge result sets.

4648. Limit result size — Cap rows/items a function returns.

Structured output specifics

4649. Define schema once — Put the schema in the system prompt, not per request.

4650. Short keys — Use compact JSON keys.

4651. Flat schemas — Avoid deep nesting overhead.

4652. Enums for fixed values — Constrain fields to enums.

4653. Omit defaults — Skip fields equal to known defaults.

4654. Arrays over repeated objects — Positional arrays where schema is shared.

4655. Bounded lists — Cap array lengths in the schema.

4656. Codes over labels — Return codes, map to text client-side.

4657. No explanation fields — Drop fields you won't use.

4658. Compact dates — ISO short forms, not verbose dates.

4659. Strict validation — Validate in code; don't ask the model to narrate.

4660. Trim string lengths — Constrain max length per field.

4661. Single object per item — Avoid wrapper overhead in batches.

4662. Reuse key dictionary — Define keys once for a batch.

Batch processing specifics

4663. One instruction per batch — Write rules once, apply to all items.

4664. Index-keyed results — Return results keyed by input index.

4665. Right-size batches — Balance batch size against context limits.

4666. Dedupe within batch — Drop duplicate items before sending.

4667. Pre-filter the batch — Remove items rules can answer without the model.

4668. Shared few-shot — One example set serves the whole batch.

4669. Partition by type — Batch by task type for the tightest instructions.

4670. Aggregate then finalize — Reduce partials to a small final pass.

4671. Skip cached items — Don't reprocess items already answered.

4672. Stream batch output — Process results as they arrive, stop early.

4673. Use the Batch API — Submit bulk jobs to discounted endpoints.

4674. Compact array output — Ask for arrays, not paragraphs.

4675. Coalesce tiny calls — Merge many small calls into periodic larger ones.

4676. Shared context once — Load common context once for the batch.

4677. Cap per-item output — Limit output length for each batched item.

4678. Parallelize batches — Run independent batches concurrently.

Retry & error handling

4679. Don't retry blindly — Fix the cause; blind retries multiply tokens.

4680. Exponential backoff — Back off to avoid retry storms.

4681. Cap retry count — Bound retries so failures don't compound cost.

4682. Idempotent calls — Avoid re-running calls that re-inject big results.

4683. Circuit breakers — Stop hammering a failing path.

4684. Validate before retry — Confirm the input is fixable before retrying.

4685. Partial-result reuse — Resume from the last good step, not from scratch.

4686. Cheap-model fallback — Fail over to a cheaper model, not a costlier retry.

4687. Schema-guided repair — Repair invalid output minimally, not by regenerating.

4688. Log retry token cost — Measure how much retries actually cost.

4689. Dead-letter, don't loop — Park failures instead of infinite retrying.

4690. Timeout tuning — Set timeouts to avoid wasted long generations.

4691. Deterministic settings — Lower temperature to reduce retry-worthy variance.

4692. Cache successful results — Never reprocess what already succeeded.

4693. Graceful degradation — Return a cheap fallback over an expensive retry.

4694. Pre-validate JSON — Catch malformed output early to avoid re-asks.


❤️ Support the project: https://saweria.co/0xgetz