{{item.videoDuration}}
{{item.title}}
{{item.text}}
{{item.videoDuration}}
{{item.text}}
The market dynamics demand IT transformation
The previous article discussed the evolution of trading and risk management strategies in the Japanese power wholesale market as market liquidity increases. For trading organisations, one of the key enablers for successful participation in the current and future market is their IT infrastructure. This article outlines the requirements for a modern trading IT landscape and examines how new technologies contribute to successful trading operations in a changing market environment.
In present-day volatile markets, global energy trading firms regularly manage a vast volume of market transactions—often numbering in the tens of thousands daily, with peaks exceeding 100,0001 transactions on certain days. Typically, these transactions originate from various markets and strategies. However, the recent surge in deal counts is predominantly attributable to activities in the intraday spot market. Transaction counts are also increasing in Japan, yet despite notable growth in recent years, the volume of intraday trades is still lower than that of comparable international power markets. In 2024, the JEPX intraday market recorded 7,390 GWh, whereas markets such as Germany and the UK handle several times this volume. When evaluated alongside the similarly sized day-ahead markets in these overseas markets, Japan’s relatively low intraday trading volume appears to be an outlier, as depicted in Figure 1.
FIGURE 1: COMPARISON OF DAY-AHEAD AND INTRADAY VOLUMES: JAPAN, GERMANY, UK2
It seems plausible that intraday volumes in Japan will pick up in the coming years, driven by Japan’s ambitious plans to increase the share of variable renewable assets from 26.7% in 20243 to 40–50% by 20404. Battery energy storage system capacity is also expected to grow to 14–24 GWh by 20305 , as both asset types heavily utilise intraday trading.
In continuous intraday markets, price information for traded products can fluctuate multiple times per second, leading to significant market volatility. Hence, effectively managing positions by integrating various data types and sources becomes increasingly complex for power traders. Continuing to rely on outdated monolithic trading systems or even managing positions via spreadsheets creates substantial operational and compliance risks in today’s operations and prevents organisations from implementing new strategies going forward. Therefore, a modern IT landscape is essential for secure operations and enables organisations to capitalise on Japan’s increasing market liquidity and volatility, such as by capturing arbitrage opportunities, lowering imbalance costs or optimising battery use and dispatch of flexible assets.
What a modern trading IT landscape needs to deliver
To meet current and future market demands, the trading IT must deliver a range of advanced capabilities in addition to supporting the traditional trade lifecycle and serving as a record of transactions. Key requirements include:
- Real-time trade execution: Systems must enable traders (or algorithms) to execute orders instantly on external marketplaces. This means providing real-time market access and low-latency order routing.
- Intraday position monitoring: In continuous markets, constant monitoring of positions and trading P&L across different markets, commodities and tenors is essential—particularly considering volatile renewable generation and flexibility needs.
- Advanced analytics and decision support: To support real-time exposure monitoring and trade execution, IT needs to provide price views, forecasts and analytical capabilities at comparable granularity. Machine learning models or AI-driven decision support for new use cases should be included or at least planned for in the IT roadmap.
- Automated end-to-end processing and integration: A modern landscape must support the full trade lifecycle in a largely automated way—from deal capture to settlement—with minimal manual steps. Asset optimisation use cases extend these requirements to the physical side, including high-performance scheduling platform and optimisation tools.
- Non-functional requirements: On top of functional capabilities, technological requirements are essential to ensure the platform’s performance and scalability over time. A key requirement is the capability to handle high data volumes from various sources—trade ticks, quotes, position updates, etc.—and store this information comprehensively and in real time. Achieving this level of performance typically necessitates cloud-based components, which in turn can facilitate compliance with evolving cybersecurity and availability standards necessary for 24/7 operations. Notably, flexibility to accommodate ongoing changes, in the form of adding new functionality or tools, and a short time to market for new business cases are mission critical.
How new technologies shape the trading landscape
Historically, the ETRM (energy trading and risk management system) served as the primary system for recording and managing deals from inception until settlement. Although it remains a key system in most organisations, its role is shifting towards only covering specialised functions, such as transaction storage and providing an audit trail for transactions. Other business functionalities are increasingly handled outside the ETRM, typically in the form of business services designed for a specific task, such as trade execution, risk calculations or reporting. This decoupling of applications facilitates a domain-driven architecture with software structured around the core business domains of energy trading (e.g. trading, risk, operations) instead of within one monolithic software. Many organisations have found that loosely coupled services (microservice architecture), despite adding some complexity, dramatically improve their ability to deploy updates frequently, implement custom features and scale or replace parts of the system selectively. Figure 2 shows a schematic IT trading landscape with a traditional ETRM at its core, surrounded by satellite services that reflect business processes and provide interfaces.
FIGURE 2: EXAMPLE TRADING IT LANDSCAPE: ETRM AS THE SYSTEM OF RECORD IN A SYSTEM OF BUSINESS SERVICES FOR TARGETED FUNCTIONALITY AND INTERFACES
At the same time, a suite of tools and services increases the volume of information—such as deal, event or price messages—transmitted between systems. Traditionally, organisations have initiated their modernisation efforts by implementing message queues to decouple applications, wherein message producers place information onto a queue for consumption by a single application. However, as interconnections become more complex and requirements increase regarding scalability and simultaneous data access by multiple consumers, many companies are transitioning to data streaming technologies as an integration layer between applications. This technology allows for real-time processing of information by providing a live feed of continuous incremental updates. Data producers publish these updates to designated ‘channels’, to which data consumers can subscribe. Data streams retain data for configurable periods, allowing for replay in support of troubleshooting, audits or onboarding of new applications. In an energy trading scenario, this might include continuously aggregating trade data to update positions by the minute or merging power plant sensor data with market prices to inform dispatch decisions. See Figure 3 for a schematic visualisation of real-time data processing by various applications, from data generation to consumption.
FIGURE 3: DATA STREAMING AS A BASIS FOR REAL-TIME DATA PROCESSING
Additionally, streaming technology is often used to orchestrate the complex suite of microservices via event-driven architecture. Instead of processes polling for updates or batch interfaces, any significant occurrence in the system is published as events on a dedicated streaming channel. For example, when a trade is executed, a ‘TradeExecuted’ event is emitted; when a new price tick comes in or a forecast is updated, those are events too. Multiple services can subscribe to these events and react as needed—decoupled from the event producers. This pattern is useful for setups where different systems need the same data simultaneously.
While data streaming stores data for a short period of time, data platforms have become the industry standard to consolidate data and provide tools for analysis, reporting, and machine learning and AI use cases. These platforms may be set up as traditional data warehouses, data lakes, data mesh systems—where each domain team manages its own data products for shared use—or through a combination of these approaches. Considering the variety of use cases and tools, comprehensive data governance and a data schema registry (governing how messages, such as deal, price or position data are structured in an organisation) are essential for ensuring the reusability of data.
Modernisation requires a clear vision and target landscape
Modernising trading operations in a live environment is certainly a long-term initiative. Delayed ETRM projects have too often cost organisations both money and competitive advantage. A clear vision of the target landscape should sit at the centre of a modernisation roadmap, guiding central technology decisions, domain boundaries and build vs. buy decisions. Undoubtedly, IT will play a central role in future trading operations. Therefore, close cooperation between business and IT teams will pave the way for a successful transformation.
While this article has provided an overview of the trading IT landscape, the next instalment in this series will examine automated and algorithmic trading processes and tools, considering not only IT aspects but also broader business implications.
2 Sources: JEPX, EPEX, Nordpool; Nordpool data is from 2022, as Nordpool has stopped publishing volume data since then. Note: Differences in market design affect comparability
(JPEX does not operate a separate intraday auction on top of the continuous intraday market, while EPEX does so in Germany and the UK).
{{filterContent.facetedTitle}}
{{contentList.dataService.numberHits}} {{contentList.dataService.numberHits == 1 ? 'result' : 'results'}}
{{contentList.loadingText}}
{{contentList.loadingText}}
