How cross-team collaboration and smart data engineering freed up 200 hours of compute time, half a petabyte of storage and valuable cluster resources - consolidating redundant processing across multiple products into one unified content pipeline!
Transform your service delivery by transitioning from manual, email-based processes to structured, automated Ivanti Request Offerings and integration with third party platforms (i.e. Jira). This session demonstrates how to standardize requests and reduce administrative overhead. Key Takeaways • Automate Processes: Eliminate manual email and paper-based request systems. • Reduce IT Workload: Lower administrative burdens through automated workflows. • Optimize Resources: Shift engineering talent away from repetitive manual fulfillment tasks. • Boost Productivity: Standardize service delivery to accelerate turnaround times. • Enhance Experience: Drive end-user satisfaction via intuitive self-service portals.
This presentation showcases how the Scientific Domain Expansion team within Content Operations transformed and modernized our approach to data ingestion, review, and harmonization—significantly increasing efficiency, scalability, and overall data quality. Previously, creating and validating new XML datasets required substantial manual effort, often involving 4–6 team members and covering less than 10% of mapped data over multiple review cycles spanning several days. Similar challenges arose when validating MarkLogic ODH updates, where achieving comprehensive data coverage through manual checks alone was not feasible.
To address these limitations, we implemented a more scalable and automated approach by leveraging ODH production data, MarkLogic scripts, and Excel-based validation techniques. These tools allow us to perform bulk checks across entire datasets, quickly identify mapping issues, and collaborate with developers to validate updates and catch errors early in the process.
As a result, tasks that once required days of effort and multiple reviewers can now be completed by a single individual in minutes to hours—delivering faster turnaround times, increased coverage, and improved confidence in data quality.
TECOPS is ramping up the next phase of automation to introduce automated upgrades for all the things. Self service will be available to enable teams fast, automated execution for common tasks. Join us as we highlight what you can expect in the upcoming months and the many benefits we will all gain as we complete this next phase of centralized operations.
Spark Weave is an orchestration layer built on top of Apache Spark that lets you define and run multi-job data pipelines as a single, managed workflow. Rather than wiring together spark-submit calls by hand, you describe a set of jobs, declare what data each one produces and consumes, and the framework handles all the other stuff like dependency resolution, parallel execution, failure isolation, and resume.
