To add a Logstash filter, place a filter { ... } section between your input and output sections, then test the configuration before sending it to production. A useful centralized pipeline commonly parses raw text with grok, sets the event time with date, normalizes fields with mutate, and expands JSON payloads with json.
Antihost offers Linux VPS plans with full root access, unlimited traffic on a 1 Gbps port and a 14-day free trial.
The filter is not a separate service. It is the shared processing stage in a pipeline that receives events from many hosts (for example, through Beats or Kafka) and forwards the resulting events to Elasticsearch or another output. The sections below show how to build that stage safely, how filter order affects the result, and how to diagnose parse failures.
As an Amazon Associate I earn from qualifying purchases.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Where the filter belongs in a centralized Logstash pipeline
Logstash configurations have three logical sections, in this order:
- Input: receives events from Beats, Kafka, files, TCP and other input plugins.
- Filter: optionally parses, enriches, renames, converts or removes event data.
- Output: writes the event to Elasticsearch, a file, stdout or another destination.
Events flow through those stages in sequence. The centralized aspect comes from having one shared pipeline receive events from many machines; filters are simply the middle stage applied to those events. You can run the same filter logic for web servers, application hosts and network devices, provided the incoming formats are known and the configuration accounts for their differences.
#1 Best Overall
- [Ultra-Lightweight Ear Buds Designed for Small Ears] Each earbud weighs only 3.7g and features a compact, ergonomic in-ear design made especially for small ears. Secure, low-profile, and comfortable for workouts, all-day wear or overnight sleeping.The ultra-slim profile sits flush in your ear with no hard edges poking or rubbing, so you can even sleep on your side without discomfort.
- [Immersive Stereo Sound with TOZO OrigX Technology] TOZO OrigX tuning delivers clear vocals, balanced mids, and natural stereo sound for music, podcasts, and videos.
- [Long Battery Life for Daily Use] Get up to 7 hours of playtime on a single charge, with up to 32 hours total using the charging case—ideal for workdays, commuting, and extended listening sessions.
- [Bluetooth 5.3 & Stable Connection] Bluetooth 5.3 provides fast pairing, stable wireless performance, and reduced dropouts as you move around home or office.
- [Deep Bass with Clear Vocals] High-performance drivers produce punchy bass while keeping vocals clean and detailed for everyday listening.
A minimal pipeline shape
input {
beats {
port => 5044
}
}
filter {
# Parsing and normalization go here
}
output {
stdout {
codec => rubydebug
}
}The Beats listener on port 5044 is a common starting point in Elastic’s Logstash documentation. Replace it with the input your deployment actually uses. The filter section is optional, so a pipeline can forward events unchanged, but unparsed messages are harder to search, aggregate and alert on.
What each filter does
Choose filters according to the shape of the event you receive. Do not apply every filter to every message without a condition: a JSON parser applied to plain text, for example, creates a parse-failure tag instead of useful fields.
| Filter | Primary job | Typical input | Result |
|---|---|---|---|
grok |
Extracts named fields from unstructured text with patterns such as TIMESTAMP_ISO8601 and LOGLEVEL. |
Apache-style lines, application text, mixed human-readable logs. | Fields such as ts, level and msg. |
date |
Parses a field containing a timestamp and assigns the event’s @timestamp. |
A field extracted by Grok or supplied by an input. | Correct event-time ordering and time-range searches. |
mutate |
Renames, converts, lowercases, strips, copies or removes fields. | Fields created by other filters or the input. | Consistent names and data types for queries and aggregations. |
json |
Expands JSON stored in a field into structured fields. | A whole-line JSON message or a JSON fragment in a field. | Nested or top-level fields that can be searched individually. |
Before you edit the pipeline
Identify the actual event shape
Capture representative events from every producer. Note whether the payload is a complete JSON object, plain text, or a mixture of formats. Record the field that contains the original payload (often message), the timestamp format, timezone, severity spelling and fields whose type must be numeric.
Keep a safe test output
During development, send output to stdout with the rubydebug codec. It displays the complete event, including fields and tags, so you can see whether parsing worked before indexing anything. Keep a copy of the last known-good configuration and change one logical stage at a time.
Check the Logstash release and bundled plugins
Plugin versions are separate from the Logstash core version. RubyGems lists logstash-filter-grok 4.4.4 (released 2026-02-11) and logstash-filter-mutate 3.6.0 (released 2026-03-26), but those are gem release dates, not a promise that every Logstash package includes those exact versions. Verify the plugins bundled with the Logstash release you run and test after upgrades.
Step 1: Extract fields with Grok
Grok matches text against named patterns and writes the captured values to fields. A pattern can contain several captures, and the match value can be an array when a source uses more than one line format.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Rank #2
- 【IP68 Waterproof Multifunctinal Smart Watch】Torjalph fitness activity tracker smart watch is IP68 waterproof technology ,you can wear it when swimming,handwashing,it's rain-proof which means that you perform multiple exercise modes outside without worrying about sweat or rain corroding your fitness smart watch,and this watch equipped with a 230 mAh large capacity battery magnetic charging ,and this sports activity tracker watch can be used 5-7 days If fully charged.
- 【Larger 1.83'' Full Screen Watch & Multi-dial Selection】1.83 inch full HD color touch screen smart watch for men / women provides excellent 240*284 resolution image quality ,This Bluetooth smart watch offers a better touch experience and remarkable immersion and ,more than 200 watch faces for you to choose from App"Da Fit ", you can change the dial style according to your mood to wear keeping the world on your wrist
- 【Accurate Activity Tracker Smart Watch 】sports watch activity tracker uses sweat and dust resistant breathable silicone strap, our Fitness Tracker Smart Watch for Android iOS Phones supports 12 sports modes,walking,running,bicycle,swimming,badminton,basketball,football,rope skipping. This bluetooth calls smart watch for men women is lightweight and stylish, bringing more convenient and relaxing sports experience while also providing more protection.
- 【Your 24 Hour Personal Daily Health Monitor】Fitness watch for women men full touch screen IP67 waterproof bluetooth smartwatch for Android iOS Phones with heart rate blood pressure monitor multifunction Andriod smart watch with steps counter sleep monitor sports fitness watch, monitoring blood oxygen saturation allows you to understand your physical condition in real time to track fitness progress ,to know your health status in time.
- 【More Functions Bluetooth Call Smart Watch】This fitness activity smart watch full screen for iPhone and Android phones includes a bunch of new features including:The bluetooth smart watch can hang up the phone, receive and display WhatsAPP /Text /Facebook /Twitter /Messenger reminder.Support sedentary reminder, alarm clock, stopwatch, timer,weather information,music control, remote camera control compatible with Android 5.0 and above or iOS 9.0 and above smartphones.
A basic application-line pattern
filter {
grok {
match => {
"message" => "%{TIMESTAMP_ISO8601:ts} %{LOGLEVEL:level} %{GREEDYDATA:msg}"
}
}
}For a line such as 2026-10-14T07:02:58Z WARN Cache miss for user 42, the pattern creates ts, level and msg. GREEDYDATA captures the remainder of the line, while the timestamp and level patterns give you reusable matches instead of a fragile character expression.
Supporting multiple line formats
filter {
grok {
match => {
"message" => [
"%{TIMESTAMP_ISO8601:ts} %{LOGLEVEL:level} %{GREEDYDATA:msg}",
"%{IP:client_ip} %{WORD:method} %{URIPATH:request} %{NUMBER:status}"
]
}
}
}Grok tries the alternatives in the order written. Put the most specific format first so a broad pattern does not consume a line that should have matched a more useful one. If a line does not match, inspect the event for Grok’s failure tag and compare the raw text with the pattern character by character.
Do not parse the wrong field
Set match to the field that actually contains the text. If an input has already placed the payload under a nested field, matching message will silently leave the intended field untouched. Use stdout debugging to confirm the field path before changing the pattern.
Step 2: Set @timestamp with the date filter
Grok extraction creates a timestamp field; it does not automatically make that value the event timestamp. The date filter parses the extracted field and writes the result to @timestamp, which is the field used for time-based searches and ordering.
Recommended Free Tools
ISO 8601 timestamps
filter {
date {
match => ["ts", "ISO8601"]
target => "@timestamp"
remove_field => ["ts"]
}
}ISO8601 handles standard ISO-formatted values, including values carrying an offset or a UTC marker. Removing ts is optional; do it only when the original field is no longer needed for troubleshooting or auditing.
Custom formats and timezones
For a non-ISO value, provide the appropriate date pattern instead of ISO8601. If the source omits a timezone, set the timezone option to the timezone in which that device or application writes its logs. Otherwise, the same wall-clock time can be interpreted differently from host to host. Do not assume that a server’s local timezone is UTC simply because the Logstash host is configured for UTC.
Rank #3
- [Low Noise High Fidelity] New bee guitar cord with 22AWG oxygen-free-copper center conductors provides enhanced signal transfer, the PVC insulation material and the high-density stranded copper double shield provide noise rejection for quiet operation
- [Ideal for Choice] New bee 1/4 straight and right angle metal connectors, ensures a secure connection in any instrument, pedal, or amp jack. L shape great fro on stage guitar stands
- [Wide Compatible] Suitable for mixer, power amplifier, microphone, effector, decoder, equalizer, electoronic organ, electric guitar, electric bass and other electronic equipments. Widely used in the installation of smart home, karaoke speaker, professional stage audio, etc
- [Easy Disassemble] Making incidental repairs quick and easy! Very nice, flexible cable
- [Dont Hesite to Order] 24 hours customer services and professional technology team are standing by
filter {
date {
match => ["ts", "yyyy-MM-dd HH:mm:ss"]
timezone => "Europe/London"
target => "@timestamp"
}
}Use the actual zone for the producer and document daylight-saving behavior for sites that span regions. If parsing fails, retain the original field until you have corrected the format; deleting it immediately removes the evidence needed to diagnose the problem.
Step 3: Normalize fields with Mutate
Use mutate after extraction and timestamp parsing to make fields predictable for dashboards and alerts.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallConvert numeric values
filter {
mutate {
convert => {
"status" => "integer"
"bytes" => "integer"
}
}
}A value that looks numeric but remains a string can sort lexicographically and behave incorrectly in aggregations. Convert only fields whose input is reliably numeric; malformed values will not become valid integers merely because a conversion was requested.
Normalize case, names and retained data
filter {
mutate {
rename => { "clientIP" => "client_ip" }
lowercase => ["level"]
remove_field => ["debug_context"]
}
}Consistent field names reduce query complexity. Keep the original field when it is needed for forensics, and remove large or redundant fields only after verifying that downstream users do not depend on them.
Understand Mutate’s internal order
Operations inside one mutate block are not executed in the order they appear. Elastic documents this fixed order: coerce, rename, update, replace, convert, gsub, uppercase, capitalize, lowercase, strip, split, join, merge and copy. For example, a rename is completed before a convert, so a field can be renamed and then converted in the same block. If you need a different dependency, use separate mutate blocks and place them in the required pipeline order.
Step 4: Expand JSON payloads
When the entire value of message is a JSON object, point the JSON filter’s source at that field:
Free tools Windows power users keep installed
One-click scans. No signup required.
Rank #4
- Digital Family Calendar: BSIMB digital calendar wall touch screen puts an end to your chaotic life. Designed for busy families, the digital calendar keeps you organised and in step with your family. Assign different colours and tasks to each family member to make family scheduling easier and more intuitive.
- Easy Setup and Auto-Sync: Plug in, connect to WLAN and auto-syn your electric calendar in minutes. Digital wall calendar compatible with Google calendar, iCloud calendar, Outlook calendar, Cozi calendar and Yahoo calendar. Add events directly on the device or through the ecalendar app. Electronic calendar stays current whether you're at work or traveling—no manual updates needed.
- All-In-One Home Assistant: Smart calendar builds habits and encourage independence with personalised chores. Display daily meal plan to reduce the daily ‘What's for dinner?’ stress. To-do lists keep track of everything you want. Turning chores into valuable achievements with stars and rewards, making life easier and more enjoyable.
- Manage Anytime Anywhere: BSIMB electronic calendar for family stay connected with the powerful ecalendar app. Access and manage your digital wall calendar anytime, anywhere with free features like stars & rewards, photo screensaver, chore charts, meals plans, to-do lists etc. Electronic planner paid features like Magic Import tailor advanced tools for users with special needs.
- Stay in Touch with Loved Ones: Share photos or videos from your phone to electronic wall calendar with app, instantly update your life moment, and keep in touch with your family, especially those who are far away; you can also surprise your smart calendar digital wall by sending him a wish card; it's also great for keeping tech-illiterate elders in touch with the youngsters.
filter {
json {
source => "message"
}
}The parsed keys become event fields. To keep the object grouped under a namespace, set target:
filter {
json {
source => "message"
target => "application"
}
}Use a conditional when one centralized pipeline receives both JSON and plain-text events. Applying json to a non-JSON line adds a JSON parse-failure tag rather than extracting fields.
filter {
if [json_logs] == true {
json {
source => "message"
target => "application"
}
}
}The condition must use a field or tag that your input actually sets. A filename, Beats field, listener port or source-specific tag can provide that discriminator. Do not copy a configuration that assumes every message is JSON when your producers are mixed.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Step 5: Chain filters in the right order
Logstash executes filter plugins in the order written. A dependable text-log chain is therefore:
- Grok: extract the timestamp and other values.
- Date: parse the extracted timestamp and set
@timestamp. - Mutate: convert types, normalize names and remove fields.
For JSON logs, parse the JSON first and then normalize the fields that were created. A single pipeline can branch by source when formats differ.
Combined starting example
filter {
if [json_logs] == true {
json {
source => "message"
target => "application"
}
} else {
grok {
match => {
"message" => "%{TIMESTAMP_ISO8601:ts} %{LOGLEVEL:level} %{GREEDYDATA:msg}"
}
}
date {
match => ["ts", "ISO8601"]
target => "@timestamp"
remove_field => ["ts"]
}
}
mutate {
convert => { "status" => "integer" }
lowercase => ["level"]
remove_field => ["message"]
}
}This is a documented-behavior starting point, not a guarantee that it matches your events. Validate the JSON discriminator, field names and timestamp format with real samples. If JSON messages also carry a timestamp that must replace @timestamp, parse that nested field with a date filter after the JSON expansion.
Step 6: Test before rollout
Run Logstash’s configuration check before starting a production pipeline:
bin/logstash -f first-pipeline.conf --config.test_and_exitA successful check means the configuration can be parsed; it does not prove that every event matches your patterns. Next, run the pipeline with stdout and rubydebug, send representative events, and inspect:
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
- Whether extracted fields contain the expected values.
- Whether
@timestampreflects the producer’s event time rather than ingestion time. - Whether numeric fields are displayed as numbers.
- Whether unwanted
_grokparsefailureor JSON parse-failure tags appear. - Whether the original
messageis retained long enough to troubleshoot.
Test at least one valid event, one malformed event and one event from each known producer format. Keep the malformed case: a central pipeline should fail visibly and predictably rather than silently indexing misleading fields.
Common failures and fixes
| Symptom | Likely cause | Fix |
|---|---|---|
| Fields are missing and a Grok failure tag appears. | The pattern targets the wrong field or does not match punctuation, spacing or format. | Inspect the raw event with rubydebug, correct the field path or add a second pattern for the producer’s format. |
@timestamp remains the ingestion time. |
The date filter did not run, used the wrong source field or used the wrong format. | Check that Grok created the timestamp field, use ISO8601 or the correct custom pattern, and verify the timezone. |
| Every plain-text event has a JSON parse-failure tag. | The JSON filter runs unconditionally on mixed input. | Guard it with a reliable field or tag that identifies JSON events. |
| A status aggregation sorts incorrectly. | The value is indexed as a string. | Convert it to an integer with mutate and verify the resulting event type. |
| A renamed field is not converted. | The conversion references the old name or the dependency is spread across blocks incorrectly. | Use the documented Mutate order, or rename in one block and convert in a following block. |
| Useful debugging data disappears. | The filter removes message or the original timestamp too early. |
Delay remove_field until parsing is verified; retain source data when audit or forensic requirements require it. |
Operating a filter in a shared pipeline
Separate format-specific logic
Centralization does not require one giant pattern. Use conditionals based on source metadata, tags or input-specific fields, then keep each branch narrow. This makes a new producer less likely to break existing formats and makes failure tags easier to interpret.
Preserve a useful event contract
Agree on names and types for fields that dashboards and alerts share: a lowercase severity field, an integer status, a consistently named client address and a real @timestamp. Keep application-specific data nested when names could collide across producers.
Roll out changes incrementally
- Copy the current configuration and add the smallest parsing change.
- Run
--config.test_and_exit. - Replay representative events to a non-production output or a temporary index.
- Compare parsed fields and failure tags with the prior version.
- Deploy to a limited source group before routing all hosts through the new filter.
- Monitor Logstash and destination errors, then retain the rollback configuration.
Plugin and documentation versions change over time. Recheck the filter plugin versions bundled with your Logstash release whenever you upgrade, especially when a configuration relies on option behavior or ordering.
Filter selection at a glance
| If your raw event looks like… | Start with… | Then apply… |
|---|---|---|
2026-10-14T07:02:58Z INFO Started |
grok |
date, then mutate |
{"level":"info","status":200} |
json |
date for any event-time field, then mutate |
| Several known text formats from different hosts | Conditional grok branches or an ordered pattern array |
date and mutate per branch as needed |
| Already structured fields with inconsistent types | No parser, unless a nested JSON string remains | mutate for rename, conversion, case and cleanup |
What success looks like
A correctly filtered event has a timestamp that reflects when the source says the event happened, searchable fields extracted from the original payload, stable names and types, and no unexpected parse-failure tags. The original payload is retained or removed deliberately, not accidentally. Most importantly, the pipeline continues to make failures observable: a line that does not match should be easy to identify and correct without guessing what Logstash did.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
