Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[InstrFDO][TypeProf] Implement binary instrumentation and profile read/write #66825

Merged
merged 44 commits into from
Apr 1, 2024
Merged
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
361f8f6
[IRPGO][ValueProfile] Instrument virtual table address that could be …
mingmingl-llvm Sep 17, 2023
cb0246f
Address feebacks on the code. Questions will be followed up shortly.
mingmingl-llvm Sep 29, 2023
3c6ca04
In InstrProf.cpp, add option -enable-vtable-type-profiling to flag co…
mingmingl-llvm Oct 2, 2023
3780bd9
A few fixes:
mingmingl-llvm Oct 2, 2023
9255867
resolve feedbacks on code
mingmingl-llvm Oct 4, 2023
8d8a45a
Rebase onto main (a fresh commit on Oct27) and add the following changes
mingmingl-llvm Oct 28, 2023
82fac8e
address feedback
mingmingl-llvm Nov 1, 2023
d54d059
small changes:
mingmingl-llvm Nov 1, 2023
ebf4c1d
pick up upstream change; sync main branch with upstream and 'git merg…
mingmingl-llvm Nov 7, 2023
52143e9
update clang-format version to 117.0.4 released binary and run clang-…
mingmingl-llvm Nov 7, 2023
2350d7f
The changes:
mingmingl-llvm Nov 7, 2023
af34929
Update main branch of my llvm fork. Run 'git merge main' and resolve …
mingmingl-llvm Nov 9, 2023
fcf92af
resolve feedback
mingmingl-llvm Nov 13, 2023
f5d12b5
merge upstream main changes and resolve conflict
mingmingl-llvm Nov 16, 2023
17d941c
Merge branch 'main' into vtable
mingmingl-llvm Nov 16, 2023
c000e64
run clang-format over modified files
mingmingl-llvm Nov 16, 2023
5b09e43
merge main branch changes
mingmingl-llvm Nov 25, 2023
313eb10
undo clang-format on unchanged lines
mingmingl-llvm Nov 26, 2023
abcbc6d
Changes:
mingmingl-llvm Dec 3, 2023
8c725ef
fix three test failures related with zlib usage
mingmingl-llvm Dec 6, 2023
1050a6b
apply git-format change
mingmingl-llvm Dec 11, 2023
9f01a30
run 'git merge main', resolve conflicts and run clang-format on the diff
mingmingl-llvm Dec 14, 2023
a9deaec
Polish tests
mingmingl-llvm Dec 14, 2023
66efde8
In raw profile reader, read multi-byte data using 'swap'. The 'swap' …
mingmingl-llvm Dec 30, 2023
9ec0784
undo git-formatted lines and follow existing style
mingmingl-llvm Jan 2, 2024
0d9abe5
run 'git merge main'
mingmingl-llvm Jan 2, 2024
8240524
one-liner fix for tools/llvm-profdata/raw-64-bits-le.test
mingmingl-llvm Jan 2, 2024
60cd296
fix two test failures. They are overlooked when running 'git merge main'
mingmingl-llvm Jan 2, 2024
a7633ad
run 'git merge main'
mingmingl-llvm Feb 27, 2024
a61c8a4
A few changes after 'git merge main':
mingmingl-llvm Feb 27, 2024
4dbe23e
Changes
mingmingl-llvm Feb 28, 2024
000d818
apply clang-format change
mingmingl-llvm Feb 28, 2024
c57422b
simple clean-ups
mingmingl-llvm Mar 2, 2024
df6eadf
For helper function needsComdatForCounter, limit the first parameter …
mingmingl-llvm Mar 3, 2024
c30888c
Merge branch 'main' into vtable
mingmingl-llvm Mar 4, 2024
4a9e6c9
unify 'maybeSetComdat'
mingmingl-llvm Mar 4, 2024
61e8292
resolve review feedback
mingmingl-llvm Mar 6, 2024
fc1d2be
resolve comment
mingmingl-llvm Mar 26, 2024
d8967d2
merge main branch and resolve conflicts
mingmingl-llvm Mar 26, 2024
27239f0
Merge branch 'main' into vtable
mingmingl-llvm Mar 28, 2024
0158523
Changes:
mingmingl-llvm Mar 28, 2024
2abdfe3
remove InstrProfSymtab::getGlobalVariable method; it's not used in th…
mingmingl-llvm Mar 28, 2024
7ea9217
resolve comments
mingmingl-llvm Mar 29, 2024
0a0c364
Changes:
mingmingl-llvm Mar 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
run 'git merge main'
  • Loading branch information
mingmingl-llvm committed Feb 27, 2024
commit a7633ad0e42e867565845accf7818fcd35921292
34 changes: 16 additions & 18 deletions compiler-rt/include/profile/InstrProfData.inc
Original file line number Diff line number Diff line change
@@ -105,14 +105,13 @@ INSTR_PROF_DATA(const uint32_t, llvm::Type::getInt32Ty(Ctx), NumBitmapBytes, \
#else
#define INSTR_PROF_VTABLE_DATA_DEFINED
#endif
INSTR_PROF_VTABLE_DATA(
const uint64_t, llvm::Type::getInt64Ty(Ctx), VTableNameHash,
ConstantInt::get(llvm::Type::getInt64Ty(Ctx),
IndexedInstrProf::ComputeHash(PGOVTableName)))
INSTR_PROF_VTABLE_DATA(const IntPtrT, llvm::Type::getInt8PtrTy(Ctx),
INSTR_PROF_VTABLE_DATA(const uint64_t, llvm::Type::getInt64Ty(Ctx), \
VTableNameHash, ConstantInt::get(llvm::Type::getInt64Ty(Ctx), \
IndexedInstrProf::ComputeHash(PGOVTableName)))
INSTR_PROF_VTABLE_DATA(const IntPtrT, llvm::PointerType::getUnqual(Ctx), \
VTablePointer, VTableAddr)
INSTR_PROF_VTABLE_DATA(const uint32_t, llvm::Type::getInt32Ty(Ctx), VTableSize,
ConstantInt::get(llvm::Type::getInt32Ty(Ctx),
INSTR_PROF_VTABLE_DATA(const uint32_t, llvm::Type::getInt32Ty(Ctx), VTableSize, \
mingmingl-llvm marked this conversation as resolved.
Show resolved Hide resolved
ConstantInt::get(llvm::Type::getInt32Ty(Ctx), \
VTableSizeVal))
#undef INSTR_PROF_VTABLE_DATA
/* INSTR_PROF_VTABLE_DATA end. */
@@ -167,8 +166,8 @@ INSTR_PROF_RAW_HEADER(uint64_t, CountersDelta,
INSTR_PROF_RAW_HEADER(uint64_t, BitmapDelta,
(uintptr_t)BitmapBegin - (uintptr_t)DataBegin)
INSTR_PROF_RAW_HEADER(uint64_t, NamesDelta, (uintptr_t)NamesBegin)
INSTR_PROF_RAW_HEADER(uint64_t, VNamesSize, VNamesSize)
INSTR_PROF_RAW_HEADER(uint64_t, NumVTables, NumVTables)
INSTR_PROF_RAW_HEADER(uint64_t, VNamesSize, VNamesSize)
INSTR_PROF_RAW_HEADER(uint64_t, ValueKindLast, IPVK_Last)
#undef INSTR_PROF_RAW_HEADER
/* INSTR_PROF_RAW_HEADER end */
@@ -210,21 +209,19 @@ VALUE_PROF_FUNC_PARAM(uint32_t, CounterIndex, Type::getInt32Ty(Ctx))
VALUE_PROF_KIND(IPVK_IndirectCallTarget, 0, "indirect call target")
/* For memory intrinsic functions size profiling. */
VALUE_PROF_KIND(IPVK_MemOPSize, 1, "memory intrinsic functions size")
/* For virtual table address profiling, the addresses of the virtual table
/* For virtual table address profiling, the address point of the virtual table
* (i.e., the address contained in objects pointing to a virtual table) are
* profiled. Note this may not be the address of the per C++ class virtual table
* object (i.e., there is an offset).
* object (e.g., there might be an offset).
*
* The profiled addresses are stored in raw profile, together with the following
* two types of information.
* 1. The (beginning and ending) addresses of per C++ class virtual table objects.
* 1. The (starting and ending) addresses of per C++ class virtual table objects.
* 2. The (compressed) virtual table object names.
* RawInstrProfReader converts profiled virtual table addresses to virtual table
* objects' MD5 hash.
*/
VALUE_PROF_KIND(IPVK_VTableTarget, 2, "The address of the compatible vtable (i.e., "
"there is an offset from this address to per C++ "
"class virtual table global variable.)")
VALUE_PROF_KIND(IPVK_VTableTarget, 2, "The profiled address point of the vtable")
/* These two kinds must be the last to be
* declared. This is to make sure the string
* array created with the template can be
@@ -306,9 +303,9 @@ COVMAP_HEADER(uint32_t, Int32Ty, Version, \
#undef COVMAP_HEADER
/* COVMAP_HEADER end. */


#ifdef INSTR_PROF_SECT_ENTRY
#define INSTR_PROF_DATA_DEFINED

INSTR_PROF_SECT_ENTRY(IPSK_data, \
INSTR_PROF_QUOTE(INSTR_PROF_DATA_COMMON), \
INSTR_PROF_DATA_COFF, "__DATA,")
@@ -352,6 +349,7 @@ INSTR_PROF_SECT_ENTRY(IPSK_covname, \
#undef INSTR_PROF_SECT_ENTRY
#endif


#ifdef INSTR_PROF_VALUE_PROF_DATA
#define INSTR_PROF_DATA_DEFINED

@@ -523,6 +521,7 @@ getValueProfRecordHeaderSize(uint32_t NumValueSites);
#undef INSTR_PROF_VALUE_PROF_DATA
#endif /* INSTR_PROF_VALUE_PROF_DATA */


#ifdef INSTR_PROF_COMMON_API_IMPL
#define INSTR_PROF_DATA_DEFINED
#ifdef __cplusplus
@@ -749,7 +748,7 @@ serializeValueProfDataFrom(ValueProfRecordClosure *Closure,
than WIN32 */
#define INSTR_PROF_DATA_COMMON __llvm_prf_data
#define INSTR_PROF_NAME_COMMON __llvm_prf_names
#define INSTR_PROF_VNAME_COMMON __llvm_prf_vtabnames
#define INSTR_PROF_VNAME_COMMON __llvm_prf_vns
#define INSTR_PROF_CNTS_COMMON __llvm_prf_cnts
#define INSTR_PROF_BITS_COMMON __llvm_prf_bits
#define INSTR_PROF_VALS_COMMON __llvm_prf_vals
@@ -765,7 +764,7 @@ serializeValueProfDataFrom(ValueProfRecordClosure *Closure,
*/
#define INSTR_PROF_DATA_COFF ".lprfd$M"
#define INSTR_PROF_NAME_COFF ".lprfn$M"
#define INSTR_PROF_VNAME_COFF ".lprfn$M"
#define INSTR_PROF_VNAME_COFF ".lprfvn$M"
#define INSTR_PROF_CNTS_COFF ".lprfc$M"
#define INSTR_PROF_BITS_COFF ".lprfb$M"
#define INSTR_PROF_VALS_COFF ".lprfv$M"
@@ -994,4 +993,3 @@ InstrProfIsSingleValRange(uint64_t Value) {
}

#endif /* INSTR_PROF_VALUE_PROF_MEMOP_API */

6 changes: 3 additions & 3 deletions compiler-rt/lib/profile/InstrProfiling.h
Original file line number Diff line number Diff line change
@@ -50,7 +50,7 @@ typedef struct ValueProfNode {
} ValueProfNode;

typedef void *IntPtrT;
typedef struct VTableProfData {
typedef struct COMPILER_RT_ALIGNAS(INSTR_PROF_DATA_ALIGNMENT) VTableProfData {
#define INSTR_PROF_VTABLE_DATA(Type, LLVMType, Name, Initializer) Type Name;
#include "profile/InstrProfData.inc"
} VTableProfData;
@@ -117,8 +117,8 @@ char *__llvm_profile_begin_bitmap(void);
char *__llvm_profile_end_bitmap(void);
ValueProfNode *__llvm_profile_begin_vnodes();
ValueProfNode *__llvm_profile_end_vnodes();
VTableProfData *__llvm_profile_begin_vtables();
VTableProfData *__llvm_profile_end_vtables();
const VTableProfData *__llvm_profile_begin_vtables();
const VTableProfData *__llvm_profile_end_vtables();
uint32_t *__llvm_profile_begin_orderfile();

/*!
22 changes: 5 additions & 17 deletions compiler-rt/lib/profile/InstrProfilingBuffer.c
Original file line number Diff line number Diff line change
@@ -83,18 +83,6 @@ uint64_t __llvm_profile_get_data_size(const __llvm_profile_data *Begin,
const __llvm_profile_data *End) {
return __llvm_profile_get_num_data(Begin, End) * sizeof(__llvm_profile_data);
}
COMPILER_RT_VISIBILITY
uint64_t __llvm_profile_get_num_vtable(const VTableProfData *Begin,
const VTableProfData *End) {
intptr_t EndI = (intptr_t)End, BeginI = (intptr_t)Begin;
return (EndI + sizeof(VTableProfData) - 1 - BeginI) / sizeof(VTableProfData);
}

COMPILER_RT_VISIBILITY
uint64_t __llvm_profile_get_vtable_section_size(const VTableProfData *Begin,
const VTableProfData *End) {
return __llvm_profile_get_num_vtable(Begin, End) * sizeof(VTableProfData);
}

// Counts the number of `VTableProfData` elements within the range of [Begin,
// End). Caller should guarantee that End points to one byte past the inclusive
@@ -236,8 +224,8 @@ uint64_t __llvm_profile_get_size_for_buffer_internal(
DataSize, CountersSize, NumBitmapBytes, NamesSize, 0 /* VTableSize */,
0 /* VNameSize */, &PaddingBytesBeforeCounters,
&PaddingBytesAfterCounters, &PaddingBytesAfterBitmapBytes,
&PaddingBytesAfterNames, NULL /* PaddingBytesAfterVTable */,
NULL /* PaddingbytesAfterVNames */);
&PaddingBytesAfterNames, &PaddingBytesAfterVTable,
&PaddingBytesAfterVNames);

return sizeof(__llvm_profile_header) + __llvm_write_binary_ids(NULL) +
DataSize + PaddingBytesBeforeCounters + CountersSize +
@@ -269,7 +257,7 @@ COMPILER_RT_VISIBILITY int __llvm_profile_write_buffer_internal(
// Set virtual table arguments to NULL since they are not supported yet.
return lprofWriteDataImpl(
&BufferWriter, DataBegin, DataEnd, CountersBegin, CountersEnd,
BitmapBegin, BitmapEnd, 0 /* VPDataReader */, NamesBegin, NamesEnd,
NULL /* VTableBegin */, NULL /* VTableEnd */, NULL /* VNamesBegin */,
NULL /* VNamesEnd */, 0 /* SkipNameDataWrite */);
BitmapBegin, BitmapEnd, /*VPDataReader=*/0, NamesBegin, NamesEnd,
/*VTableBegin=*/NULL, /*VTableEnd=*/NULL, /*VNamesBegin=*/NULL,
/*VNamesEnd=*/NULL, /*SkipNameDataWrite=*/0);
}
12 changes: 5 additions & 7 deletions compiler-rt/lib/profile/InstrProfilingMerge.c
Original file line number Diff line number Diff line change
@@ -107,14 +107,13 @@ static uintptr_t signextIfWin64(void *V) {
#endif
}

// Skip names section, vtable profile data section and vtable names section
// for runtime profile merge. To merge runtime addresses from multiple
// profiles collected from the same instrumented binary, the binary should be
// loaded at fixed base address (e.g., build with -no-pie, or run with ASLR
// disabled). In this set-up these three sections remain unchanged.
static uint64_t
getDistanceFromCounterToValueProf(const __llvm_profile_header *const Header) {
// Skip names section, vtable profile data section and vtable names section
// for runtime profile merge. To merge runtime addresses from multiple
// profiles collected from the same instrumented binary, the binary should be
// loaded at fixed base address (e.g., build with -no-pie, or run with ASLR
// disabled).
// In this set-up these three sections remain unchanged.
const uint64_t VTableSectionSize =
Header->NumVTables * sizeof(VTableProfData);
const uint64_t PaddingBytesAfterVTableSection =
@@ -157,7 +156,6 @@ int __llvm_profile_merge_from_buffer(const char *ProfileData,
Header->NumCounters * __llvm_profile_counter_entry_size();
SrcBitmapStart = SrcCountersEnd;
SrcNameStart = SrcBitmapStart + Header->NumBitmapBytes;

SrcValueProfDataStart =
SrcNameStart + getDistanceFromCounterToValueProf(Header);
if (SrcNameStart < SrcCountersStart || SrcNameStart < SrcBitmapStart)
5 changes: 3 additions & 2 deletions compiler-rt/lib/profile/InstrProfilingPlatformLinux.c
Original file line number Diff line number Diff line change
@@ -77,10 +77,11 @@ COMPILER_RT_VISIBILITY const char *__llvm_profile_begin_vtabnames(void) {
COMPILER_RT_VISIBILITY const char *__llvm_profile_end_vtabnames(void) {
return &PROF_VNAME_STOP;
}
COMPILER_RT_VISIBILITY VTableProfData *__llvm_profile_begin_vtables(void) {
COMPILER_RT_VISIBILITY const VTableProfData *
__llvm_profile_begin_vtables(void) {
return &PROF_VTABLE_START;
}
COMPILER_RT_VISIBILITY VTableProfData *__llvm_profile_end_vtables(void) {
COMPILER_RT_VISIBILITY const VTableProfData *__llvm_profile_end_vtables(void) {
return &PROF_VTABLE_STOP;
}
COMPILER_RT_VISIBILITY char *__llvm_profile_begin_counters(void) {
1 change: 0 additions & 1 deletion llvm/include/llvm/Analysis/IndirectCallVisitor.h
Original file line number Diff line number Diff line change
@@ -12,7 +12,6 @@
#ifndef LLVM_ANALYSIS_INDIRECTCALLVISITOR_H
#define LLVM_ANALYSIS_INDIRECTCALLVISITOR_H

#include "llvm/ADT/SetVector.h"
#include "llvm/IR/InstVisitor.h"
#include <vector>

4 changes: 2 additions & 2 deletions llvm/include/llvm/ProfileData/InstrProf.h
Original file line number Diff line number Diff line change
@@ -582,7 +582,7 @@ class InstrProfSymtab {
AddrToMD5Map.push_back(std::make_pair(Addr, MD5Val));
}

/// Map the address range (i.e., [start_address, end_address]) of a variable
/// Map the address range (i.e., [start_address, end_address)) of a variable
/// to its names' MD5 hash. This interface is only used by the raw profile
/// reader.
void mapVTableAddress(uint64_t StartAddr, uint64_t EndAddr, uint64_t MD5Val) {
@@ -1177,7 +1177,7 @@ struct Header {
uint64_t MemProfOffset;
uint64_t BinaryIdOffset;
uint64_t TemporalProfTracesOffset;
uint64_t VTableNamesOffset; // Organize virtual table names.
uint64_t VTableNamesOffset;
// New fields should only be added at the end to ensure that the size
// computation is correct. The methods below need to be updated to ensure that
// the new field is read correctly.
40 changes: 29 additions & 11 deletions llvm/include/llvm/ProfileData/InstrProfData.inc
Original file line number Diff line number Diff line change
@@ -96,19 +96,22 @@ INSTR_PROF_DATA(const uint32_t, llvm::Type::getInt32Ty(Ctx), NumBitmapBytes, \
#undef INSTR_PROF_DATA
/* INSTR_PROF_DATA end. */

/* For a virtual table object, record the name hash to associate profiled
* addresses with global variables, and record {starting address, size in bytes}
* to map the profiled virtual table (which usually have an offset from the
* starting address) back to a virtual table object. */
#ifndef INSTR_PROF_VTABLE_DATA
#define INSTR_PROF_VTABLE_DATA(Type, LLVMType, Name, Initializer)
#else
#define INSTR_PROF_VTABLE_DATA_DEFINED
#endif
INSTR_PROF_VTABLE_DATA(
const uint64_t, llvm::Type::getInt64Ty(Ctx), VTableNameHash,
ConstantInt::get(llvm::Type::getInt64Ty(Ctx),
IndexedInstrProf::ComputeHash(PGOVTableName)))
INSTR_PROF_VTABLE_DATA(const IntPtrT, llvm::PointerType::getUnqual(Ctx),
INSTR_PROF_VTABLE_DATA(const uint64_t, llvm::Type::getInt64Ty(Ctx), \
VTableNameHash, ConstantInt::get(llvm::Type::getInt64Ty(Ctx), \
IndexedInstrProf::ComputeHash(PGOVTableName)))
INSTR_PROF_VTABLE_DATA(const IntPtrT, llvm::PointerType::getUnqual(Ctx), \
VTablePointer, VTableAddr)
INSTR_PROF_VTABLE_DATA(const uint32_t, llvm::Type::getInt32Ty(Ctx), VTableSize,
ConstantInt::get(llvm::Type::getInt32Ty(Ctx),
INSTR_PROF_VTABLE_DATA(const uint32_t, llvm::Type::getInt32Ty(Ctx), VTableSize, \
ConstantInt::get(llvm::Type::getInt32Ty(Ctx), \
VTableSizeVal))
#undef INSTR_PROF_VTABLE_DATA
/* INSTR_PROF_VTABLE_DATA end. */
@@ -163,8 +166,8 @@ INSTR_PROF_RAW_HEADER(uint64_t, CountersDelta,
INSTR_PROF_RAW_HEADER(uint64_t, BitmapDelta,
(uintptr_t)BitmapBegin - (uintptr_t)DataBegin)
INSTR_PROF_RAW_HEADER(uint64_t, NamesDelta, (uintptr_t)NamesBegin)
INSTR_PROF_RAW_HEADER(uint64_t, VNamesSize, VNamesSize)
INSTR_PROF_RAW_HEADER(uint64_t, NumVTables, NumVTables)
INSTR_PROF_RAW_HEADER(uint64_t, VNamesSize, VNamesSize)
INSTR_PROF_RAW_HEADER(uint64_t, ValueKindLast, IPVK_Last)
#undef INSTR_PROF_RAW_HEADER
/* INSTR_PROF_RAW_HEADER end */
@@ -206,7 +209,19 @@ VALUE_PROF_FUNC_PARAM(uint32_t, CounterIndex, Type::getInt32Ty(Ctx))
VALUE_PROF_KIND(IPVK_IndirectCallTarget, 0, "indirect call target")
/* For memory intrinsic functions size profiling. */
VALUE_PROF_KIND(IPVK_MemOPSize, 1, "memory intrinsic functions size")
VALUE_PROF_KIND(IPVK_VTableTarget, 2, "vtable target")
/* For virtual table address profiling, the address point of the virtual table
* (i.e., the address contained in objects pointing to a virtual table) are
* profiled. Note this may not be the address of the per C++ class virtual table
* object (e.g., there might be an offset).
*
* The profiled addresses are stored in raw profile, together with the following
* two types of information.
* 1. The (starting and ending) addresses of per C++ class virtual table objects.
* 2. The (compressed) virtual table object names.
* RawInstrProfReader converts profiled virtual table addresses to virtual table
* objects' MD5 hash.
*/
VALUE_PROF_KIND(IPVK_VTableTarget, 2, "The profiled address point of the vtable")
/* These two kinds must be the last to be
* declared. This is to make sure the string
* array created with the template can be
@@ -288,6 +303,7 @@ COVMAP_HEADER(uint32_t, Int32Ty, Version, \
#undef COVMAP_HEADER
/* COVMAP_HEADER end. */


#ifdef INSTR_PROF_SECT_ENTRY
#define INSTR_PROF_DATA_DEFINED
INSTR_PROF_SECT_ENTRY(IPSK_data, \
@@ -333,6 +349,7 @@ INSTR_PROF_SECT_ENTRY(IPSK_covname, \
#undef INSTR_PROF_SECT_ENTRY
#endif


#ifdef INSTR_PROF_VALUE_PROF_DATA
#define INSTR_PROF_DATA_DEFINED

@@ -372,7 +389,7 @@ typedef struct ValueProfRecord {
/*!
* Return the number of value sites.
*/
uint32_t getNumValueSites() const { return NumValueSites; }
uint32_t getNumValueSites() const { return NumValueSites; }
/*!
* Read data from this record and save it to Record.
*/
@@ -504,6 +521,7 @@ getValueProfRecordHeaderSize(uint32_t NumValueSites);
#undef INSTR_PROF_VALUE_PROF_DATA
#endif /* INSTR_PROF_VALUE_PROF_DATA */


#ifdef INSTR_PROF_COMMON_API_IMPL
#define INSTR_PROF_DATA_DEFINED
#ifdef __cplusplus
@@ -730,7 +748,7 @@ serializeValueProfDataFrom(ValueProfRecordClosure *Closure,
than WIN32 */
#define INSTR_PROF_DATA_COMMON __llvm_prf_data
#define INSTR_PROF_NAME_COMMON __llvm_prf_names
#define INSTR_PROF_VNAME_COMMON __llvm_prf_vtabnames
#define INSTR_PROF_VNAME_COMMON __llvm_prf_vns
#define INSTR_PROF_CNTS_COMMON __llvm_prf_cnts
#define INSTR_PROF_BITS_COMMON __llvm_prf_bits
#define INSTR_PROF_VALS_COMMON __llvm_prf_vals
7 changes: 3 additions & 4 deletions llvm/include/llvm/ProfileData/InstrProfReader.h
Original file line number Diff line number Diff line change
@@ -666,13 +666,12 @@ class IndexedInstrProfReader : public InstrProfReader {
std::unique_ptr<MemProfRecordHashTable> MemProfRecordTable;
/// MemProf frame profile data on-disk indexed via frame id.
std::unique_ptr<MemProfFrameHashTable> MemProfFrameTable;
/// The reader itself doesn't decompress vtable names. A compiler that reads
/// indexed profiles could construct symtab from module IR so it doesn't need
/// the decompressed names.
/// VTableNamePtr points to the beginning of compressed vtable names.
/// When a symtab is constructed from profiles by llvm-profdata, the list of
/// names could be decompressed based on `VTableNamePtr` and
/// `CompressedVTableNamesLen`.
/// VTableNamePtr points to the beginning of compressed vtable names.
/// A compiler that reads indexed profiles could construct symtab from module
/// IR so it doesn't need the decompressed names.
const char *VTableNamePtr = nullptr;
/// The length of compressed vtable names.
uint64_t CompressedVTableNamesLen = 0;
19 changes: 8 additions & 11 deletions llvm/lib/ProfileData/InstrProf.cpp
Original file line number Diff line number Diff line change
@@ -391,13 +391,12 @@ std::string getPGOName(const GlobalVariable &V, bool InLTO) {
return getIRPGOObjectName(V, InLTO, nullptr /* PGONameMetadata */);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: /PGONameMetaData=/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

}

// See getIRPGOFuncName() for a discription of the format.
std::pair<StringRef, StringRef>
getParsedIRPGOFuncName(StringRef IRPGOFuncName) {
auto [FileName, FuncName] = IRPGOFuncName.split(';');
if (FuncName.empty())
return std::make_pair(StringRef(), IRPGOFuncName);
return std::make_pair(FileName, FuncName);
// See getIRPGOObjectName() for a discription of the format.
std::pair<StringRef, StringRef> getParsedIRPGOName(StringRef IRPGOName) {
auto [FileName, MangledName] = IRPGOName.split(kGlobalIdentifierDelimiter);
if (MangledName.empty())
return std::make_pair(StringRef(), IRPGOName);
return std::make_pair(FileName, MangledName);
}

StringRef getFuncNameWithoutPrefix(StringRef PGOFuncName, StringRef FileName) {
@@ -561,10 +560,7 @@ Error InstrProfSymtab::initVTableNamesFromCompressedStrings(
std::bind(&InstrProfSymtab::addVTableName, this, std::placeholders::_1));
}

Error InstrProfSymtab::addFuncWithName(Function &F, StringRef PGOFuncName) {
if (Error E = addFuncName(PGOFuncName))
return E;
MD5FuncMap.emplace_back(Function::getGUID(PGOFuncName), &F);
StringRef InstrProfSymtab::getCanonicalName(StringRef PGOName) {
// In ThinLTO, local function may have been promoted to global and have
// suffix ".llvm." added to the function name. We need to add the
// stripped function name to the symbol table so that we can find a match
@@ -1657,6 +1653,7 @@ size_t Header::size() const {
case 12ull:
return offsetOf(&Header::VTableNamesOffset) +
sizeof(Header::VTableNamesOffset);
[[fallthrough]];
case 11ull:
[[fallthrough]];
case 10ull:
Loading
You are viewing a condensed version of this merge commit. You can view the full changes here.