Power servers are multi-processor systems built on the NUMA (Non-Uniform Memory Access) architecture.
In this architecture, each processor has a directly attached portion of RAM and can also access the entire server memory (including memory attached to other processors). Processor has the fastest access to memory directly attached to it (local memory), while memory assigned to other processors takes longer to access (non-local memory).
Types of Memory in Power Servers
In Power servers, we identify three types of memory (listed from fastest to slowest access):
- Local – memory directly attached to the processor
- Near – memory attached to another processor within the same CEC
- Far – memory attached to a processor in a different CEC
For scale-up servers like the E980/1080, all three types of memory are available. However, in smaller servers like the S922, only local and far memory are present.
Optimal situation occurs when LPAR uses memory directly assigned to processor on which it operates. When LPAR is created, the hypervisor attempts to assign resources in the most efficient way, but in some situations, it’s not always possible. Even LPAR that had optimally assigned RAM may begin operating suboptimally during its runtime.
Operations on the server, such as:
- LPM
- Creating LPAR
- DLPAR
can impact on memory assigments to LPAR and make it operating suboptimally.
Dynamic Platform Optimizer (DPO)
Starting from POWER7 servers, the hypervisor supports Dynamic Platform Optimizer (DPO) function, which allows for processor/memory affinity optimization. The DPO function is initiated from the HMC console, running it rearranges the logical partition’s processor/memory affinity.
During DPO operation, it’s important to note that some virtualization functions may be temporarily blocked, such as:
- LPM
- DLPAR
If any blocked operations are required, DPO process can be interrupted.
When running DPO, please note that changing the processor/memory affinity may temporarily affect performance due to memory relocation.
Two Steps in Memory Optimization
Memory optimization is performed in two steps:
1. Planning:
Planning includes assessing the current memory optimization and determining whether improvements can be made.
To check the current memory optimization level, run the following command:
lsmemopt -m <SYSTEM> -r <sys/lpar> -o currscore
For example, to check the system’s current memory score:
lsmemopt -m Power-LAB1 -r sys -o currscore curr_sys_score=89
For more detailed information for LPARs, use:
admin@hmclab:~> lsmemopt -m Power-LAB1 -r lpar -o currscore lpar_name=VIOS 1 LAB1,lpar_id=1,curr_lpar_score=100 lpar_name=VIOS 2 LAB1,lpar_id=2,curr_lpar_score=100 lpar_name=TEST AIX1,lpar_id=3,curr_lpar_score=100 lpar_name=TEST AIX2,lpar_id=4,curr_lpar_score=45 lpar_name=AIX 71,lpar_id=6,curr_lpar_score=100 lpar_name=AIX 73,lpar_id=7,curr_lpar_score=100 lpar_name=AIX LKU TEST,lpar_id=9,curr_lpar_score=20 lpar_name=AIX 72,lpar_id=12,curr_lpar_score=100 lpar_name=AIX 72 N2,lpar_id=25,curr_lpar_score=90
The result will show the DPO value – “curr_lpar_score“, where 0 is the worst score, and 100 is the best.
To estimate whether memory affinitty can be improved, use:
lsmemopt -m <SYSTEM> -r lpar -o calcscore
where “predicted_lpar_score” indicates predicted DPO value
admin@hmclab:~> lsmemopt -m Power-LAB1 -r lpar -o calcscore lpar_name=VIOS 1 LAB1,lpar_id=1,curr_lpar_score=100,predicted_lpar_score=100 lpar_name=VIOS 2 LAB1,lpar_id=2,curr_lpar_score=100,predicted_lpar_score=100 lpar_name=TEST AIX1,lpar_id=3,curr_lpar_score=100,predicted_lpar_score=100 lpar_name=TEST AIX2,lpar_id=4,curr_lpar_score=45,predicted_lpar_score=100 lpar_name=AIX 71,lpar_id=6,curr_lpar_score=100,predicted_lpar_score=100 lpar_name=AIX 73,lpar_id=7,curr_lpar_score=100,predicted_lpar_score=100 lpar_name=AIX LKU TEST,lpar_id=9,curr_lpar_score=20,predicted_lpar_score=100 lpar_name=AIX 72,lpar_id=12,curr_lpar_score=100,predicted_lpar_score=100 lpar_name=AIX 72 N2,lpar_id=25,curr_lpar_score=90,predicted_lpar_score=100
2. Optimization:
Once the planning phase is completed, the next step is the actual optimization of memory.
To start the optimization process, use the following command:
optmem -m <SYSTEM> -o start -t affinity
You can monitor progress during the process with:
lsmemopt -m <SYSTEM>
admin@hmclab:~> lsmemopt -m Power-LAB1 in_progress=1,status=In progress,type=affinity,opt_id=110,progress=0,requested_lpar_ids=none,protected_lpar_ids=none,impacted_lpar_ids=none admin@hmclab:~> lsmemopt -m Power-LAB1 in_progress=1,status=In progress,type=affinity,opt_id=110,progress=39,requested_lpar_ids=none,protected_lpar_ids=none,impacted_lpar_ids=none admin@hmclab:~> lsmemopt -m Power-LAB1 in_progress=0,status=Finished,type=affinity,opt_id=10,progress=100,requested_lpar_ids=none,protected_lpar_ids=none,"impacted_lpar_names=VIOS 1 LAB1,VIOS 2 LAB1,TEST AIX1,TEST AIX2,AIX 71,AIX 73,AIX LKU TEST,AIX 72,AIX 72 N2","impacted_lpar_ids=1,2,3,4,6,7,9,12,25"
To stop optimization, use:
optmem -m <SYSTEM> -o stop
During estimation, following switches can be helpful:
–id – Specifies ID of one or more partitions to prioritize during optimization.
–xid – Specifies ID of one or more partitions to exclude from optimization and keep protected.
Sometimes, achieving optimal results requires performing several calculations to find the right combination of –id and –xid.
When optimizing, it’s important to remember that powerred-off LPARs are also optimized. Therefore, you can exclude them from the optimization process, which can help shorten its duration.
After completing the optimization, recheck the current memory score to ensure that the optimization was successful. Sometimes, optimization may be performed, but the results differ from the calculations.
Best Practices for Memory Optimization
Memory optimization in Power servers is dynamic and should be monitored.
To achieve the best results, consider these tips:
- Optimize Excluded LPARs: Even if powered-off LPARs are excluded from optimization, remember that they may still hold resources, blocking them for others. Consider clearing resources from powered-off LPARs to ensure the optimal use of resources.
- Use DPO: Run Dynamic Platform Optimizer to optimize processor memory affinity.
- Monitor Optimization: Continuously monitor the system’s memory optimization.