Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[lldb] Add SBType::FindDirectNestedType() function #68705

Merged
merged 15 commits into from
Oct 14, 2023

Conversation

Endilll
Copy link
Contributor

@Endilll Endilll commented Oct 10, 2023

This patch adds a SBType::FindDirectNestedType(name) function which performs a non-recursive search in given class for a type with specified name. The intent is to perform a fast search in debug info, so that it can be used in formatters, and let them remain responsive.

This is driven by my work on formatters for Clang and LLVM types. In particular, by PointerIntPairInfo::MaskAndShiftConstants, which is required to extract pointer and integer from PointerIntPair.

Related Discourse thread: https://discourse.llvm.org/t/traversing-member-types-of-a-type/72452

@llvmbot
Copy link
Member

llvmbot commented Oct 10, 2023

@llvm/pr-subscribers-lldb

Author: Vlad Serebrennikov (Endilll)

Changes

This patch adds a SBType::FindNestedType(name) function which performs a non-recursive search in given class for a type with specified name. The intent is to perform a fast search in debug info, so that it can be used in formatters, and let them remain responsive.

This is driven by my work on formatters for Clang and LLVM types. In particular, by PointerIntPairInfo::MaskAndShiftConstants, which is required to extract pointer and integer from PointerIntPair.

Related Discourse thread: https://discourse.llvm.org/t/traversing-member-types-of-a-type/72452


Full diff: https://github.com/llvm/llvm-project/pull/68705.diff

9 Files Affected:

  • (modified) lldb/bindings/interface/SBTypeDocstrings.i (+7)
  • (modified) lldb/include/lldb/API/SBType.h (+2)
  • (modified) lldb/include/lldb/Symbol/Type.h (+2)
  • (modified) lldb/include/lldb/Symbol/TypeSystem.h (+4)
  • (modified) lldb/source/API/SBType.cpp (+9)
  • (modified) lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp (+4)
  • (modified) lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.h (+2)
  • (modified) lldb/source/Symbol/Type.cpp (+13)
  • (modified) lldb/source/Symbol/TypeSystem.cpp (+4)
diff --git a/lldb/bindings/interface/SBTypeDocstrings.i b/lldb/bindings/interface/SBTypeDocstrings.i
index 96421a6aa20104b..b4ec67da957c7d4 100644
--- a/lldb/bindings/interface/SBTypeDocstrings.i
+++ b/lldb/bindings/interface/SBTypeDocstrings.i
@@ -720,6 +720,13 @@ SBType supports the eq/ne operator. For example,::
     "
 ) lldb::SBType::GetTypeFlags;
 
+%feature("docstring",
+    "Searches for a nested type that has provided name.
+
+    Returns the type if it was found.
+    Returns invalid type if nothing was found."
+) lldb::SBType::FindNestedType;
+
 %feature("docstring",
 "Represents a list of :py:class:`SBType` s.
 
diff --git a/lldb/include/lldb/API/SBType.h b/lldb/include/lldb/API/SBType.h
index 5962f0c50dee14f..fa02197ff8f3940 100644
--- a/lldb/include/lldb/API/SBType.h
+++ b/lldb/include/lldb/API/SBType.h
@@ -215,6 +215,8 @@ class SBType {
   bool GetDescription(lldb::SBStream &description,
                       lldb::DescriptionLevel description_level);
 
+  lldb::SBType FindNestedType(const char *name);
+
   lldb::SBType &operator=(const lldb::SBType &rhs);
 
   bool operator==(lldb::SBType &rhs);
diff --git a/lldb/include/lldb/Symbol/Type.h b/lldb/include/lldb/Symbol/Type.h
index 046501931d211a7..6da4aaba401fe14 100644
--- a/lldb/include/lldb/Symbol/Type.h
+++ b/lldb/include/lldb/Symbol/Type.h
@@ -313,6 +313,8 @@ class TypeImpl {
   bool GetDescription(lldb_private::Stream &strm,
                       lldb::DescriptionLevel description_level);
 
+  CompilerType FindNestedType(ConstString name);
+
 private:
   bool CheckModule(lldb::ModuleSP &module_sp) const;
   bool CheckExeModule(lldb::ModuleSP &module_sp) const;
diff --git a/lldb/include/lldb/Symbol/TypeSystem.h b/lldb/include/lldb/Symbol/TypeSystem.h
index eb6e453e1aec0d0..b503b66eb528c68 100644
--- a/lldb/include/lldb/Symbol/TypeSystem.h
+++ b/lldb/include/lldb/Symbol/TypeSystem.h
@@ -135,6 +135,10 @@ class TypeSystem : public PluginInterface,
 
   virtual lldb::LanguageType DeclContextGetLanguage(void *opaque_decl_ctx) = 0;
 
+  // CompilerType functions
+
+  virtual CompilerDeclContext GetCompilerDeclContextForType(const CompilerType& type);
+
   // Tests
 #ifndef NDEBUG
   /// Verify the integrity of the type to catch CompilerTypes that mix
diff --git a/lldb/source/API/SBType.cpp b/lldb/source/API/SBType.cpp
index ee5b6447428098e..7fe1836ea5d670b 100644
--- a/lldb/source/API/SBType.cpp
+++ b/lldb/source/API/SBType.cpp
@@ -586,6 +586,15 @@ lldb::TemplateArgumentKind SBType::GetTemplateArgumentKind(uint32_t idx) {
   return eTemplateArgumentKindNull;
 }
 
+SBType SBType::FindNestedType(const char *name) {
+  LLDB_INSTRUMENT_VA(this);
+
+  if (!IsValid())
+    return SBType();
+  auto ret = SBType(m_opaque_sp->FindNestedType(ConstString(name)));
+  return ret;
+}
+
 SBTypeList::SBTypeList() : m_opaque_up(new TypeListImpl()) {
   LLDB_INSTRUMENT_VA(this);
 }
diff --git a/lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp b/lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp
index 69cff0f35ae4ab2..b4bf3d3fdb20c1e 100644
--- a/lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp
+++ b/lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp
@@ -2636,6 +2636,10 @@ TypeSystemClang::GetDeclContextForType(const CompilerType &type) {
   return GetDeclContextForType(ClangUtil::GetQualType(type));
 }
 
+CompilerDeclContext TypeSystemClang::GetCompilerDeclContextForType(const CompilerType& type) {
+  return CreateDeclContext(GetDeclContextForType(type));
+}
+
 /// Aggressively desugar the provided type, skipping past various kinds of
 /// syntactic sugar and other constructs one typically wants to ignore.
 /// The \p mask argument allows one to skip certain kinds of simplifications,
diff --git a/lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.h b/lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.h
index 0544de3cd33befb..806ff64ef0af76b 100644
--- a/lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.h
+++ b/lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.h
@@ -219,6 +219,8 @@ class TypeSystemClang : public TypeSystem {
 
   static clang::DeclContext *GetDeclContextForType(const CompilerType &type);
 
+  CompilerDeclContext GetCompilerDeclContextForType(const CompilerType &type) override;
+
   uint32_t GetPointerByteSize() override;
 
   clang::TranslationUnitDecl *GetTranslationUnitDecl() {
diff --git a/lldb/source/Symbol/Type.cpp b/lldb/source/Symbol/Type.cpp
index 66284eb73cad038..724973b1fd9bd06 100644
--- a/lldb/source/Symbol/Type.cpp
+++ b/lldb/source/Symbol/Type.cpp
@@ -1082,6 +1082,19 @@ bool TypeImpl::GetDescription(lldb_private::Stream &strm,
   return true;
 }
 
+CompilerType TypeImpl::FindNestedType(ConstString name) {
+  auto type_system = GetTypeSystem(false);
+  auto *symbol_file = type_system->GetSymbolFile();
+  auto decl_context = type_system->GetCompilerDeclContextForType(m_static_type);
+  llvm::DenseSet<lldb_private::SymbolFile *> searched_symbol_files;
+  TypeMap search_result;
+  symbol_file->FindTypes(name, decl_context, /*max_matches*/ 1, searched_symbol_files, search_result);
+  if (search_result.Empty()) {
+    return CompilerType();
+  }
+  return search_result.GetTypeAtIndex(0)->GetFullCompilerType();
+}
+
 bool TypeMemberFunctionImpl::IsValid() {
   return m_type.IsValid() && m_kind != lldb::eMemberFunctionKindUnknown;
 }
diff --git a/lldb/source/Symbol/TypeSystem.cpp b/lldb/source/Symbol/TypeSystem.cpp
index 24f20293056501f..ce24e312f4f35e0 100644
--- a/lldb/source/Symbol/TypeSystem.cpp
+++ b/lldb/source/Symbol/TypeSystem.cpp
@@ -186,6 +186,10 @@ std::optional<llvm::json::Value> TypeSystem::ReportStatistics() {
   return std::nullopt;
 }
 
+CompilerDeclContext TypeSystem::GetCompilerDeclContextForType(const CompilerType& type) {
+  return CompilerDeclContext();
+}
+
 #pragma mark TypeSystemMap
 
 TypeSystemMap::TypeSystemMap() : m_mutex(), m_map() {}

@github-actions
Copy link

github-actions bot commented Oct 10, 2023

✅ With the latest revision this PR passed the C/C++ code formatter.

@Endilll
Copy link
Contributor Author

Endilll commented Oct 10, 2023

Intended usage is going to look like the following for PointerIntPair:

class PointerIntPairProvider:
    def update(self):
        pointer_info = self.valobj.type.template_args[4] # PointerIntPairInfo
        masks = pointer_info.FindNestedType("MaskAndShiftConstants")
        if masks.IsValid():
            for enumerator in enum.enum_members:
                if enumerator.name == "PointerBitMask":
                    pointer_mask = enumerator.unsigned
                    # extract pointer using the mask

Fully qualified name of MaskAndShiftConstants used in clang::QualType is the following: llvm::PointerIntPairInfo<llvm::PointerUnion<const clang::Type*, const clang::ExtQuals *>, 3, llvm::PointerLikeTypeTraits<llvm::PointerUnion<const clang::Type *, const clang::ExtQuals *> > >::MaskAndShiftConstants. I'm not sure if existing SBTarget::FindFirstType() can handle this case as well as proposed function.

I'll submit a proper API test shortly.

@Endilll
Copy link
Contributor Author

Endilll commented Oct 10, 2023

It should be noted that after calling FindNestedType and successfully finding MaskAndShiftConstants in the example above, the following routine in my fork reports that PointerIntPairInfo has 2 enums instead of 1:

uint32_t TypeSystemClang::GetNumMemberEnums(lldb::opaque_compiler_type_t type) {
  using EnumIt = clang::DeclContext::specific_decl_iterator<clang::EnumDecl>;
  if (!type)
    return 0;

  clang::QualType qual_type = RemoveWrappingTypes(GetCanonicalQualType(type));
  if (GetCompleteQualType(&getASTContext(), qual_type)) {
    const clang::RecordType *record_type =
        llvm::cast<clang::RecordType>(qual_type.getTypePtr());
    const clang::RecordDecl *record_decl = record_type->getDecl();
    assert(record_decl);
    auto result = std::distance(EnumIt(record_decl->decls_begin()), EnumIt(record_decl->decls_end()));
    return result;
  }
  return 0;
}

It's also available here: https://github.com/Endilll/llvm-project/blob/fbad2d1fd8e9c67e4de8a196df0cd1d1788fa990/lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp#L7080-L7095

@jimingham
Copy link
Collaborator

jimingham commented Oct 10, 2023

The idea seems good. If you are looking for a type and you know it is defined inside another type, then providing a way for you to tell lldb that is useful.

Is FindNestedType guaranteed to have a single return? What if I had:

class A {
  class B {
      class D {
      };
   };
   class E {
      class D {
      };
    };
};


Then I would expect

class_a_type = lldb.target.FindFirstType("A")
class_a_type.FindNestedType("D")

to return both A::B::D and A::E::D. So you might need to return a SBTypeList here? You could also add a FindFirstWhatever API to return the first hit in case there may be many, though whenever I use that in anything but interactive SB API uses I feel like I probably should be being more careful, so I'm of two minds about that...

@Endilll
Copy link
Contributor Author

Endilll commented Oct 10, 2023

Is FindNestedType guaranteed to have a single return?

Yes, because search is not recursive, i.e. doesn't search inside nested types.
I'm intentionally specifying FindNestedType this way, because I'd like it to not do too much to remain fast by design, and I'm yet to see a use case when I'd be interested in type NN, buried inside arbitrary number of intervening nested types. Even if there is such use case, SBTarget::FindFirstType could be a better fit to address it.

@jimingham
Copy link
Collaborator

jimingham commented Oct 10, 2023 via email

@Endilll
Copy link
Contributor Author

Endilll commented Oct 10, 2023

depth parameter makes sense to me.
I wonder if it should have a default value 0 meaning infinite depth.

Now I wonder how my current implementation behaves with respect to depth, which is actually embarrassing. And how to implement depth parameter.

@jimingham
Copy link
Collaborator

0 to mean unlimited search sounds good to me.

lldb/source/API/SBType.cpp Outdated Show resolved Hide resolved
lldb/source/Symbol/Type.cpp Outdated Show resolved Hide resolved
lldb/source/Symbol/Type.cpp Outdated Show resolved Hide resolved
lldb/include/lldb/Symbol/Type.h Outdated Show resolved Hide resolved
lldb/source/API/SBType.cpp Outdated Show resolved Hide resolved
@Endilll
Copy link
Contributor Author

Endilll commented Oct 11, 2023

@jimingham

bool CompilerDeclContext::IsContainedInLookup(CompilerDeclContext other) const {
if (!IsValid())
return false;
// If the other context is just the current context, we don't need to go
// over the type system to know that the lookup is identical.
if (this == &other)
return true;
return m_type_system->DeclContextIsContainedInLookup(m_opaque_decl_ctx,
other.m_opaque_decl_ctx);
}

and
bool TypeSystemClang::DeclContextIsContainedInLookup(
void *opaque_decl_ctx, void *other_opaque_decl_ctx) {
auto *decl_ctx = (clang::DeclContext *)opaque_decl_ctx;
auto *other = (clang::DeclContext *)other_opaque_decl_ctx;
do {
// A decl context always includes its own contents in its lookup.
if (decl_ctx == other)
return true;
// If we have an inline namespace, then the lookup of the parent context
// also includes the inline namespace contents.
} while (other->isInlineNamespace() && (other = other->getParent()));
return false;
}

suggest that depth of the search is type-system-dependent, and that Clang type system doesn't do recursive check, save for inline namespaces, which doesn't matter for this PR. I don't mind writing an interface for recursive search and saying somewhere that depths other that 1 are not yet supported, but I consider it out of scope of this PR (and my use case) to implement recursive search. Does this sound good to you, or you'd prefer something along the lines of FindDirectNestedType?

@jimingham
Copy link
Collaborator

Just to make sure I understand. You're using FindTypes in a DeclContext using max_matches == 1. But FindTypes will recurse in the DeclContext, so depending on the actual search implementation, you could have a situation like:

class A {
  class B {
       class C {
       };
   };
  class C {
  };
};

where if we do depth first we could start recursing down B, find A::B::C which is a match, note max_matches == 1 and return. So this is really just FindFirstNestedType, I don't think you can make any claim about directness given how this is implemented.

I still think the idea of allowing a type search to be limited to a given type context is useful. We usually do "FindWhatever" returning a list, and "FindFirstWhatever" returning some not described algorithm's chosen "first" match. It's a little odd to add the "First" w/o the list version, however.

@Endilll
Copy link
Contributor Author

Endilll commented Oct 11, 2023

find A::B::C which is a match, note max_matches == 1 and return

Looking at DeclContextIsContainedInLookup, A::B::C is not a match, because its DeclContext is not DeclContext of A. This is of course is not future-proof against changes to DeclContextIsContainedInLookup that can make it smarter in this regard, but we can have something along the lines TypeSystem::DeclContextIsDirectlyContainedInLookup if we deem it necessary.

For the context, this is the first check search algorithm performs while determining a match:

if (!DIEInDeclContext(parent_decl_ctx, die))

return decl_ctx.IsContainedInLookup(actual_decl_ctx);

@jimingham
Copy link
Collaborator

Yes, if this API is going to claim to only find directly contained types, there needs to be an explicit guarantee that that's what the underlying API's do. The API you call should state that it is doing that, and you also need to write tests that make sure we only see direct types and not ones deeper in the hierarchy.

@Endilll
Copy link
Contributor Author

Endilll commented Oct 11, 2023

It's not clear to me whether TypeSystem::DeclContextIsContainedInLookup provides such a guarantee. Implementation suggests that it's only interested in direct DeclContext, maybe ignoring transparent decl contexts like inline namespace it already handles. I hope reviewers can clarify this.

@jimingham
Copy link
Collaborator

It's not clear to me whether TypeSystem::DeclContextIsContainedInLookup provides such a guarantee. Implementation suggests that it's only interested in direct DeclContext, maybe ignoring transparent decl contexts like inline namespace it already handles. I hope reviewers can clarify this.

It might be good to ask this as a separate question on the lldb project Discourse (https://discourse.llvm.org/c/subprojects/lldb/8). That's likely to get more notice than a comment deep in a patch review.

@Michael137
Copy link
Member

Michael137 commented Oct 12, 2023

It's not clear to me whether TypeSystem::DeclContextIsContainedInLookup provides such a guarantee. Implementation suggests that it's only interested in direct DeclContext, maybe ignoring transparent decl contexts like inline namespace it already handles. I hope reviewers can clarify this.

We use DeclContextIsContainedInLookup to make sure that if, e.g., a user ran expr A::B::C, then the type we find for C is in fact a child of B which is a child of A. So it really is just a way to walk up a context hierarchy directly and check that the hierarchy in DWARF matches the hierarchy in clang::DeclContext. As you say, it also supports inline namespaces, but that only came later as we needed it to make namespaces work during expression evaluation. It seems to me that this function developed organically with the needs of the expression evaluator/type lookup, and its purpose was always really to just do a directed is-contained check. While I don't think this will change (since lookup is already fragile and some features probably rely on it being this way), it would make sense to document this in the DeclContextIsContainedInLookup contract and add tests for it so it's future proof. But that could be done outside of this PR in my opinion.

@jimingham
Copy link
Collaborator

As long as it does (and will continue doing) what it says, FindDirectNestedType is okay.

Part of me thinks: "later on somebody will want to add an N levels deep search -requiring a depth parameter, and then we'll have to add another FindNestedTypeAtDepth API where passing depth of 1 will do exactly the same thing as FindDirectNestedType, unnecessarily widening the API surface."

But adding a depth parameter now to future-proof the API and only supporting 1 (leaving others to fill in the rest of the functionality) also seems a bit weak.

So I end up not having a strong feeling either way, and the functionality is useful.

Copy link
Member

@Michael137 Michael137 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, if @jimingham doesn't have any other comments

@Endilll Endilll changed the title [lldb] Add SBType::FindNestedType() function [lldb] Add SBType::FindDirectNestedType() function Oct 13, 2023
Copy link
Member

@JDevlieghere JDevlieghere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with some small nits.

lldb/source/API/SBType.cpp Outdated Show resolved Hide resolved
lldb/bindings/interface/SBTypeDocstrings.i Outdated Show resolved Hide resolved
lldb/include/lldb/Symbol/TypeSystem.h Outdated Show resolved Hide resolved
Copy link
Member

@JDevlieghere JDevlieghere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants