正在加载内容...

963963 Chat Review Portal Independent coverage of news

Seven Things to Check Before Choosing Observability

By David Kim · · 1336 words
Seven Things to Check Before Choosing Observability

In practice, crawl budget behaves differently: If a metric has no owner, it will drift until it causes an incident. The cheapest optimisation is usually removing work nobody asked for. The same reasoning holds for crawl budget. For crawl budget, the constraint matters more than the feature list. Aggregating at write time trades flexibility for predictable read cost.

In practice, data pipelines behaves differently: A queue smooths spikes but also hides how far behind you are. Retries without jitter turn a small outage into a large one. The same reasoning holds for data pipelines. For data pipelines, the constraint matters more than the feature list. Separating the reads from the writes buys room to change either side.

Release Process: The interesting number is not the average, it is the 99th percentile. Release Process: Adding a cache in front of a slow query is a fix; fixing the query is a cure. Release Process: Every abstraction you add is a place where behaviour can differ from intent.

Teams working on search indexing usually discover this the hard way. Serving static bytes is the cheapest thing you can do at the edge. A schema is an interface; changing it is a migration, not an edit. This is most visible in search indexing. Consider search indexing specifically. Track the denominator as carefully as the numerator.

For storage tiers, the constraint matters more than the feature list. If a metric has no owner, it will drift until it causes an incident. Teams working on storage tiers usually discover this the hard way. The cheapest optimisation is usually removing work nobody asked for. Aggregating at write time trades flexibility for predictable read cost. This is most visible in storage tiers.

Search Indexing: A design that cannot be rolled back is a design that cannot be changed safely. Search Indexing: Latency budgets are easier to defend when every hop has a stated ceiling. Search Indexing: Caching helps only until the invalidation rules become the bottleneck.

For access control, the constraint matters more than the feature list. Periodic jobs should be safe to run twice, because they will be. Teams working on access control usually discover this the hard way. You rarely need a new component to fix a boundary problem. The signal you want is often already logged, just not aggregated. This is most visible in access control.

If the rollback plan needs a meeting, it is not a rollback plan. That applies to edge caching as well. In practice, edge caching behaves differently: Small pages that stay small are easier to keep fast than large ones made fast. Write the invariant down; otherwise it lives only in someone's memory. The same reasoning holds for edge caching.

Log Analysis: You can often replace a coordination problem with an idempotency key. Log Analysis: Anything that grows without a bound will eventually hit one. Log Analysis: Documentation that is not tested tends to describe the previous version.

Check charging contacts and ports for moisture before reconnecting power. Do not charge a wet product, and do not insert a charging cable or plug into a damp port. If the manual gives a drying interval or a specific cleaning procedure for the port, follow it. For products with a cord, inspect the cable and connector for fraying, looseness or corrosion before each charging session; stop using a damaged charger and check the maker’s replacement guidance.

Load Balancing: A queue smooths spikes but also hides how far behind you are. Load Balancing: Retries without jitter turn a small outage into a large one. Load Balancing: Separating the reads from the writes buys room to change either side.

A design that cannot be rolled back is a design that cannot be changed safely. The same reasoning holds for queue design. For queue design, the constraint matters more than the feature list. Latency budgets are easier to defend when every hop has a stated ceiling. Teams working on queue design usually discover this the hard way. Caching helps only until the invalidation rules become the bottleneck.

The interesting number is not the average, it is the 99th percentile. That applies to release process as well. In practice, release process behaves differently: Adding a cache in front of a slow query is a fix; fixing the query is a cure. Every abstraction you add is a place where behaviour can differ from intent. The same reasoning holds for release process.

If the rollback plan needs a meeting, it is not a rollback plan. The same reasoning holds for schema markup. For schema markup, the constraint matters more than the feature list. Small pages that stay small are easier to keep fast than large ones made fast. Teams working on schema markup usually discover this the hard way. Write the invariant down; otherwise it lives only in someone's memory.

In practice, api design behaves differently: The first thing to settle is the failure mode, not the happy path. Measurements taken once are anecdotes; you need a baseline that repeats. The same reasoning holds for api design. For api design, the constraint matters more than the feature list. Costs usually concentrate in a small number of operations, so find those first.

Cloud Infrastructure: A design that cannot be rolled back is a design that cannot be changed safely. Cloud Infrastructure: Latency budgets are easier to defend when every hop has a stated ceiling. Cloud Infrastructure: Caching helps only until the invalidation rules become the bottleneck.

Store clean, dry products in a dust-free place and follow the manufacturer’s advice about keeping materials apart. Some surfaces can pick up lint or be affected by contact with other materials, so individual storage bags or compartments may be useful if the care guide recommends separation. Inspect for cracks, chips, sticky or peeling coatings, damaged seams and changes in surface texture. These signs can shorten a product’s useful life even when the underlying material is durable.

Storage Tiers: A design that cannot be rolled back is a design that cannot be changed safely. Storage Tiers: Latency budgets are easier to defend when every hop has a stated ceiling. Storage Tiers: Caching helps only until the invalidation rules become the bottleneck.

API Design: A queue smooths spikes but also hides how far behind you are. API Design: Retries without jitter turn a small outage into a large one. API Design: Separating the reads from the writes buys room to change either side.

Cost Controls: Configurations should be reviewable in a diff, not only in a console. The best time to add an index is before the table gets large. That applies to cost controls as well. In practice, cost controls behaves differently: Failures are usually correlated, so plan for the shared dependency.

Use direct language and describe the limit in practical terms. For example: “I want to use a condom every time we have sex,” or “Please ask before taking or sharing photos of me.” A person can briefly explain why, but they do not have to prove that a boundary is reasonable. If the limit is not yet clear to them, they can say so and ask to pause while they decide.

Check again when the activity changes or when someone’s response is difficult to interpret. A simple question can make room for an honest answer: “Do you want to keep going?” If the answer is uncertain, stop and give the person space. Hesitation is not an invitation to persuade them.

If a metric has no owner, it will drift until it causes an incident. This is most visible in content delivery. Consider content delivery specifically. The cheapest optimisation is usually removing work nobody asked for. Content Delivery: Aggregating at write time trades flexibility for predictable read cost.

Cost Controls: The interesting number is not the average, it is the 99th percentile. Cost Controls: Adding a cache in front of a slow query is a fix; fixing the query is a cure. Cost Controls: Every abstraction you add is a place where behaviour can differ from intent.

Related reading