Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Finish Global Dynamic Memory and Allocator Concepts #310

Merged
merged 57 commits into from
Nov 8, 2018
Merged
Show file tree
Hide file tree
Changes from 52 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
b686a7f
introducted local memory spaces
Feb 26, 2017
95dd2eb
Introducing allocator traits [no ci]
fuchsto Mar 1, 2017
4b0314c
Merge branch 'development' of github.com:dash-project/dash into feat-…
fuchsto Mar 4, 2017
6095f1c
Documentation of allocator- and memory space concepts, intermediate […
fuchsto Mar 4, 2017
286fa93
Documentation of allocator- and memory space concepts, intermediate […
fuchsto Mar 5, 2017
1858ddb
Merge branch 'development' into feat-303-glob-dyn-alloc
fuchsto Mar 6, 2017
48e6d01
Merge remote-tracking branch 'origin/development' into feat-umap-rede…
Mar 7, 2017
d663220
added Hash Policy for UnorderedMap
Mar 12, 2017
8e99f4a
added a simple memory pool for efficient allocation and deallocation
Mar 12, 2017
d6978c6
fixed includes after moving GlobDynamicMem
Mar 13, 2017
9badabc
added a small unit test for SimpleMemoryPool
Mar 13, 2017
fc7bb81
Merge remote-tracking branch 'origin/feat-303-glob-dyn-alloc' into fe…
Mar 13, 2017
3bdc29f
Merge branch 'development' into feat-303-glob-dyn-alloc
fuchsto Mar 13, 2017
be9fc50
added documentation for SimpleMemoryPool
Mar 13, 2017
a5560e0
Merge remote-tracking branch 'origin/feat-303-glob-dyn-alloc' into fe…
Mar 13, 2017
78740c5
introducing scoped local and global memory allocators
Mar 16, 2017
1f555c1
temporarily disable unit tests
Mar 17, 2017
77759eb
integrate Memory Spaces Concept in Dynmic Allocator
Mar 17, 2017
35849ee
Merge branch 'development' into feat-303-glob-dyn-alloc
fuchsto Mar 17, 2017
ad1930b
integrated allocator traits and memory spaces
Mar 17, 2017
eb7f671
Merge remote-tracking branch 'origin/feat-303-glob-dyn-alloc' into fe…
Mar 17, 2017
e7b9d14
Merge branch 'development' into feat-303-glob-dyn-alloc
fuchsto Mar 18, 2017
bf55465
Merge branch 'feat-globptr' of github.com:dash-project/dash into feat…
fuchsto Mar 18, 2017
207dd98
Merge branch 'feat-303-glob-dyn-alloc' of github.com:dash-project/das…
fuchsto Mar 18, 2017
7d1de17
STL-compliant definition of MemorySpace, added documentation and notes
fuchsto Mar 19, 2017
8c1fc03
Make Allocators aware of the underlying memory spaces
Mar 19, 2017
14b7046
Merge branch 'development' into feat-303-glob-dyn-alloc
fuchsto Mar 23, 2017
65960db
Merge branch 'feat-umap-redesign' into feat-303-glob-dyn-alloc
Mar 19, 2017
5654ea9
Merge remote-tracking branch 'origin/feat-303-glob-dyn-alloc' into fe…
Mar 23, 2017
5ac9da7
Fix as reported by Clang
Mar 23, 2017
085a1b3
Fixes for clang-5
fuchsto Mar 23, 2017
8959212
Merge remote-tracking branch 'origin/feat-303-glob-dyn-alloc' into fe…
Mar 23, 2017
94b9314
Introduced dash::max_align_t
fuchsto Mar 23, 2017
a9d71b2
Merge branch 'development' into feat-303-glob-dyn-alloc
fuchsto Mar 23, 2017
288ab7f
Merge branch 'development' of github.com:dash-project/dash into feat-…
fuchsto Mar 24, 2017
3c9b21c
Merge branch 'feat-303-glob-dyn-alloc' of github.com:dash-project/das…
fuchsto Mar 24, 2017
7a3181a
Merge branch 'development' into feat-303-glob-dyn-alloc
fuchsto Mar 24, 2017
6ffe94f
Merge branch 'development' into feat-303-glob-dyn-alloc
fuchsto Mar 24, 2017
c893c2c
Merge branch 'development' into feat-303-glob-dyn-alloc
fuchsto Mar 24, 2017
379eb3c
Reduce communication overhead in GlobHeapMem
Mar 26, 2017
673c8a1
Merge branch 'feat-globptr' into feat-303-glob-dyn-globptr-alloc
Mar 28, 2017
f83955c
Minor fixes
Mar 29, 2017
57ab07b
minor fixes in the allocators
Mar 30, 2017
c611916
Merge remote-tracking branch 'origin/development' into feat-303-glob-…
Mar 30, 2017
6cd0184
fixed resize of attached memory chunks
Mar 30, 2017
14f809d
fixed GlobalHeapMemTest
Mar 30, 2017
1ffd812
minor changes
Mar 31, 2017
a3b6dd0
Merge branch 'development' into feat-303-glob-dyn-alloc
fuchsto Apr 1, 2017
c48b280
addresses review comments
Apr 11, 2017
833e114
Merge branch 'development' into feat-303-glob-dyn-alloc
fuchsto Apr 18, 2017
2006dab
Merge branch 'development' into feat-303-glob-dyn-alloc
fuchsto Apr 24, 2017
6c08616
Merge branch 'development' into feat-303-glob-dyn-alloc
fuchsto Apr 27, 2017
31dac7d
Merge branch 'development' into feat-303-glob-dyn-alloc
fuchsto May 23, 2017
29c12bf
Merge remote-tracking branch 'origin/development' into feat-303-glob-…
Aug 7, 2017
0ff1cbe
Merge remote-tracking branch 'origin/development' into feat-303-glob-…
Aug 22, 2017
9f79019
Memory concept: use only local memory domain
Aug 25, 2017
2afc898
symmtric allocator: use dynamic windows
Aug 25, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 28 additions & 28 deletions dart-if/include/dash/dart/if/dart_globmem.h
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@
extern "C" {
#endif

#include <dash/dart/if/dart_types.h>
#include <dash/dart/if/dart_team_group.h>

/** \cond DART_HIDDEN_SYMBOLS */
#define DART_INTERFACE_ON
/** \endcond */
Expand Down Expand Up @@ -57,23 +60,43 @@ extern "C" {

*/

/*
PROPOSAL: use 128 bit global pointers with the following layout:
/*
PROPOSAL: use 128 bit global pointers with the following layout:


0 1 2 3 4 5 6 7
0123456701234567012345670123456701234567012345670123456701234567
|----<24 bit unit id>---|-flags-|-<segment id>--|---<team id>--|
|-----------<64 bit virtual address or offset>-----------------|
*/

*/
/**
* Test for NULL global pointer
*
* \ingroup DartGlobMem
*/
#define DART_GPTR_ISNULL(gptr_) \
(gptr_.unitid<0 && gptr_.segid==0 && \
gptr_.teamid==DART_TEAM_NULL && \
gptr_.addr_or_offs.addr==0)

/**
* Compare two global pointers
*
* \ingroup DartGlobMem
*/
#define DART_GPTR_EQUAL(gptr1_, gptr2_ ) \
((gptr1_.unitid == gptr2_.unitid) && \
(gptr1_.segid == gptr2_.segid) && \
(gptr1_.teamid == gptr2_.teamid) && \
(gptr1_.addr_or_offs.offset == \
gptr2_.addr_or_offs.offset) )
/**
* DART Global pointer type.
*
* \ingroup DartGlobMem
*/
typedef struct
typedef struct dart_gptr
{
/**
* The unit holding the memory element.
Expand All @@ -90,7 +113,7 @@ typedef struct
union
{
uint64_t offset;
void* addr;
void *addr;
} addr_or_offs;
} dart_gptr_t;

Expand All @@ -109,27 +132,6 @@ typedef struct
.addr_or_offs.offset = 0 }
#endif

/**
* Test for NULL global pointer
*
* \ingroup DartGlobMem
*/
#define DART_GPTR_ISNULL(gptr_) \
(gptr_.unitid<0 && gptr_.segid==0 && \
gptr_.teamid==DART_TEAM_NULL && \
gptr_.addr_or_offs.addr==0)

/**
* Compare two global pointers
*
* \ingroup DartGlobMem
*/
#define DART_GPTR_EQUAL(gptr1_, gptr2_ ) \
((gptr1_.unitid == gptr2_.unitid) && \
(gptr1_.segid == gptr2_.segid) && \
(gptr1_.teamid == gptr2_.teamid) && \
(gptr1_.addr_or_offs.offset == \
gptr2_.addr_or_offs.offset) )


/**
Expand Down Expand Up @@ -408,7 +410,6 @@ dart_ret_t dart_team_memregister(
*/
dart_ret_t dart_team_memderegister(dart_gptr_t gptr) DART_NOTHROW;


/** \cond DART_HIDDEN_SYMBOLS */
#define DART_INTERFACE_OFF
/** \endcond */
Expand All @@ -418,4 +419,3 @@ dart_ret_t dart_team_memderegister(dart_gptr_t gptr) DART_NOTHROW;
#endif

#endif /* DART_GLOBMEM_H_INCLUDED */

5 changes: 5 additions & 0 deletions dash/include/dash/Allocator.h
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
#ifndef DASH__ALLOCATOR_H__INCLUDED
#define DASH__ALLOCATOR_H__INCLUDED

#include <dash/allocator/AllocatorTraits.h>

#include <dash/allocator/LocalAllocator.h>
#include <dash/allocator/SymmetricAllocator.h>
#include <dash/allocator/EpochSynchronizedAllocator.h>


#include <dash/memory/SimpleMemoryPool.h>

namespace dash {

/**
Expand Down
15 changes: 5 additions & 10 deletions dash/include/dash/GlobPtr.h
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,6 @@ std::ostream & operator<<(
std::ostream & os,
const dart_gptr_t & dartptr);

bool operator==(
const dart_gptr_t & lhs,
const dart_gptr_t & rhs);

bool operator!=(
const dart_gptr_t & lhs,
const dart_gptr_t & rhs);

namespace dash {

// Forward-declarations
Expand Down Expand Up @@ -489,6 +481,9 @@ class GlobPtr
return _rbegin_gptr.unitid == luid.id;
}

constexpr explicit operator bool() const noexcept {
return !DART_GPTR_ISNULL(_rbegin_gptr);
}
private:
void increment(index_type offs) {
auto ptr_offset = _rbegin_gptr.addr_or_offs.offset / sizeof(value_type);
Expand Down Expand Up @@ -558,7 +553,7 @@ std::ostream & operator<<(

/**
* Internal type.
*
*
* Specialization of \c dash::GlobPtr, wraps underlying global address
* as global const pointer (not pointer to const).
*
Expand Down Expand Up @@ -838,7 +833,7 @@ dash::gptrdiff_t distance(
* \complexity O(1)
*
* \ingroup Algorithms
*
*
* \concept{DashMemorySpaceConcept}
*/
template<typename ElementType, class MemSpaceT>
Expand Down
4 changes: 2 additions & 2 deletions dash/include/dash/GlobRef.h
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ class GlobRef
* of \c operator=(const self_t &).
*/
GlobRef(const self_t & other) = delete;

/**
* Unlike native reference types, global reference types are moveable.
*/
Expand Down Expand Up @@ -180,7 +180,7 @@ class GlobRef
template <class GlobRefT>
constexpr bool operator==(const GlobRefT & other) const noexcept
{
return _gptr == other._gptr;
return DART_GPTR_EQUAL(_gptr, other._gptr);
}

template <class GlobRefT>
Expand Down
18 changes: 15 additions & 3 deletions dash/include/dash/Types.h
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
#ifndef DASH__TYPES_H_
#define DASH__TYPES_H_

#include <array>
#include <type_traits>

#include <dash/dart/if/dart_types.h>
#include <dash/internal/Unit.h>

#include <array>
#include <cstddef>
#include <type_traits>


namespace dash {

Expand Down Expand Up @@ -323,6 +324,17 @@ constexpr team_unit_t UNDEFINED_TEAM_UNIT_ID{DART_UNDEFINED_UNIT_ID};
*/
constexpr global_unit_t UNDEFINED_GLOBAL_UNIT_ID{DART_UNDEFINED_UNIT_ID};


typedef
#if defined(__clang__)
// clang has issues finding a definition for std::max_align_t,
// usually it is synonymous with the largest scalar type.
long double
#else
typename std::max_align_t
#endif
max_align_t;

} // namespace dash

#endif // DASH__TYPES_H_
26 changes: 24 additions & 2 deletions dash/include/dash/UnorderedMap.h
Original file line number Diff line number Diff line change
Expand Up @@ -182,9 +182,9 @@ class UnorderedMap
typedef typename glob_mem_type::const_reverse_local_iterator
const_reverse_local_node_iterator;

typedef typename glob_mem_type::global_iterator
typedef typename glob_mem_type::local_iterator
local_node_pointer;
typedef typename glob_mem_type::const_global_iterator
typedef typename glob_mem_type::const_local_iterator
const_local_node_pointer;

typedef UnorderedMapGlobIter<Key, Mapped, Hash, Pred, Alloc>
Expand Down Expand Up @@ -550,6 +550,28 @@ class UnorderedMap
/// The element to insert.
const value_type & value);

/**
* inserts value, using hint as a non-binding suggestion to where the search should start.
*
* Iterator validity:
*
* - All iterators in the container remain valid after the insertion unless
* it forces a rehash. In this case, all iterators in the container are
* invalidated.
* - A rehash is forced if the new container size after the insertion
* operation would increase above its capacity threshold.
* - References to elements in the map container remain valid in all cases,
* even after a rehash.
*
* \return an iterator to the inserted element, or to the element that prevented the
* insertion.
*/
iterator insert(
//Iterator hint
const_iterator hint,
//The element to insert
const value_type & value);

/**
* Insert elements in iterator range of key-value pairs, increasing the
* container size by the number of elements in the range.
Expand Down
Empty file.
69 changes: 69 additions & 0 deletions dash/include/dash/allocator/AllocatorTraits.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
#ifndef DASH__ALLOCATOR__ALLOCATOR_TRAITS_H__INCLUDED
#define DASH__ALLOCATOR__ALLOCATOR_TRAITS_H__INCLUDED

#include <dash/memory/MemorySpace.h>

#include <memory>

namespace dash {

struct collective_allocator_tag {
};

struct noncollective_allocator_tag {
};

template <class Allocator>
struct allocator_traits {

typedef Allocator allocator_type;

typedef typename allocator_type::allocator_category allocator_category;
typedef typename allocator_type::local_pointer local_pointer;

//
typedef typename allocator_type::value_type value_type;
typedef typename allocator_type::pointer pointer;

/*
typedef
typename dash::pointer_traits<pointer>::template rebind<const value_type>
const_pointer;
typedef typename dash::pointer_traits<pointer>::template rebind<void>
void_pointer;
typedef typename dash::pointer_traits<pointer>::template rebind<const void>
const_void_pointer;
*/

typedef typename allocator_type::const_pointer const_pointer;
typedef typename allocator_type::void_pointer void_pointer;
typedef typename allocator_type::const_void_pointer const_void_pointer;

typedef typename allocator_type::difference_type difference_type;
typedef typename allocator_type::size_type size_type;

/*
typedef
typename dash::pointer_traits<pointer>::difference_type difference_type;
typedef typename std::make_unsigned<difference_type>::type size_type;
*/

template <class U>
using rebind_alloc = typename allocator_type::template rebind<U>::other;

template <class U>
using rebind_traits =
dash::allocator_traits<rebind_alloc<U>>;


static pointer allocate(allocator_type& a, size_type n)
{ return a.allocate(n); }

static void deallocate(allocator_type& a, pointer p, size_type n)
{ a.deallocate(p, n); }

};

} // namespace dash

#endif // DASH__ALLOCATOR__ALLOCATOR_TRAITS_H__INCLUDED
Loading