Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #1286: String Collections support in $select. #1282

Open
wants to merge 36 commits into
base: release-8.x
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
857aefb
Checking for primitive collection in SelectExpandBinder.ProjectAsWrap…
anasik Jul 16, 2024
ea34a17
Checking only for string since that seems to be the only case that's …
anasik Jul 17, 2024
3c2ab5e
tests hehe
anasik Jul 20, 2024
79727fa
Added SQLite DB to aid with testing; Discovered that all collections …
anasik Jul 25, 2024
c126703
Added target project, excluded db file from compilation, made test co…
anasik Jul 25, 2024
cfd9ae0
Added DateOnly and put #if_NET8 around the if block. Also removed the…
anasik Jul 25, 2024
411cf73
removed commented code
anasik Jul 25, 2024
7fc15a8
Uncorrupt
anasik Jul 25, 2024
5c12e84
git attributes, last ditch effort to fix sqlite db
anasik Jul 25, 2024
85be658
Forgot to rename in some places
anasik Jul 25, 2024
2c9a286
db again fingers crossed
anasik Jul 25, 2024
77e9ec0
Removed toList from controller since the bug pertains to deferred exe…
anasik Jul 25, 2024
e638c4d
Commented out DateTimeOffset filter test cases since the error itself…
anasik Jul 25, 2024
d9c2a05
Update src/Microsoft.AspNetCore.OData/Query/Expressions/SelectExpandB…
anasik Aug 9, 2024
8d1b843
Update test/Microsoft.AspNetCore.OData.E2E.Tests/Lists/ListsDataModel.cs
anasik Aug 9, 2024
e4ea1d8
Update test/Microsoft.AspNetCore.OData.E2E.Tests/Lists/ListsControlle…
anasik Aug 9, 2024
ac53066
renamed ListTestOrders to ListTestOrder for consistency
anasik Aug 9, 2024
e75b803
Init order
anasik Aug 9, 2024
629a7b9
ADDED tests for ListTestOrder
anasik Aug 9, 2024
b780b77
TypeHelper.isPrimitive
anasik Aug 9, 2024
5946b66
Orders table
anasik Aug 9, 2024
861beda
Using direct table reference instead of Set. Dropping Orders table in…
anasik Aug 9, 2024
380c97a
Added Orders to Context
anasik Aug 9, 2024
ed8b203
Added Orders to EDM
anasik Aug 9, 2024
c086e15
Added ListTestOrder to _map and $expand to both test cases
anasik Aug 9, 2024
8d617fa
removed unnecessary spaces
anasik Aug 9, 2024
e41fb01
Update src/Microsoft.AspNetCore.OData/Query/Expressions/SelectExpandB…
anasik Aug 9, 2024
706f1eb
Renamed function. Added TimeOnly.
anasik Aug 9, 2024
2275cc2
Added Decimal
anasik Aug 9, 2024
6cbec7a
Return doc.
anasik Aug 9, 2024
d8ae66e
Update src/Microsoft.AspNetCore.OData/Common/TypeHelper.cs
anasik Aug 9, 2024
6105b07
Remove duplicated doc
anasik Aug 9, 2024
18ce012
Added copyright comment to ListsContext and removed commented lines f…
anasik Aug 10, 2024
0039e0a
Removed *.db file in favor of in-memory SQLite
anasik Aug 15, 2024
41ebcb0
removed db binary gitattribute
anasik Aug 19, 2024
167b3e8
fixed indentation
anasik Sep 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
*.sql binary
*.nupkg binary
*.exe binary
*.db binary
anasik marked this conversation as resolved.
Show resolved Hide resolved

*.ascx text
*.cd text
Expand Down
20 changes: 20 additions & 0 deletions src/Microsoft.AspNetCore.OData/Common/TypeHelper.cs
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,26 @@ Type collectionInterface
return false;
}

/// <summary>
/// Check whether the given type is a primitive type or known type.
/// </summary>
/// <param name="type">The type to validate.</param>
/// <returns>True if type is primitive or known type, otherwise False.</returns>
public static bool IsPrimitiveOrKnownType(Type type)
Copy link

@WanjohiSammy WanjohiSammy Sep 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The primitive types covered here are not sufficient. You can refer to IsPrimitiveType(Type clrType) in ODL which contains all the types that we consider primitive both Nullable or Non-Nullable primitive types. Also looking at this method, you will notice that we do not consider Uri primitive type.

Please use it to rewrite this method.

Copy link
Author

@anasik anasik Sep 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WanjohiSammy, I am well-aware of the correct primitive types but I had to cover all the types that are not only affected by the bug but were also documented to support Primitive Collections in the EF Core 8 change log.

That being said, I took a look at said method it's not immediately evident to me exactly which types of mine you're concerned about. It appears that the only difference between that method and mine is that I explicitly added cases for some types that the other function might still support.

I am almost certain that there's not a single type mentioned there that wasn't first confirmed to be affected.

Since you've already mentioned Uri,

  • Here it is being given express treatment therefore I had to support it.
  • It's also affected by the bug at hand.
  • In fact, this type is actually the most buggy of all, as I have mentioned several times in earlier interactions. In fact, I was hoping this would get merged quickly so I could debug Uri more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the confusion here stems from the fact that these two methods have similar names but serve very different purposes. The core IsPrimitiveType() test for types that can be mapped to EDM primitive types. But this method checks for types that should be treated as primitive collections in the context of EF core, without regards to EDM types. Maybe using a different name, more specific to this use case would alleviate the confusion. But I think it's fine to restrict this method to only the types that matter with respect to the EF core bug.

{
return type.IsPrimitive
|| type == typeof(string)
|| type == typeof(Uri)
|| type == typeof(DateTime)
#if NET6_0_OR_GREATER
|| type == typeof(DateOnly)
|| type == typeof(TimeOnly)
#endif
|| type == typeof(DateTimeOffset)
|| type == typeof(Guid)
|| type == typeof(Decimal);
}

internal static bool IsDictionary(Type clrType)
{
if (clrType == null)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,13 @@ internal Expression ProjectAsWrapper(QueryBinderContext context, Expression sour
int? modelBoundPageSize = null)
{
Type elementType;
bool isCollection = TypeHelper.IsCollection(source.Type, out elementType);
bool isCollection, isPrimitiveCollection = false;
isCollection = TypeHelper.IsCollection(source.Type, out elementType);

if (isCollection)
{
isPrimitiveCollection = TypeHelper.IsPrimitiveOrKnownType(elementType);
}
QueryBinderContext subContext = new QueryBinderContext(context, context.QuerySettings, elementType);
if (computeClause != null && IsAvailableODataQueryOption(context.QuerySettings, AllowedQueryOptions.Compute))
{
Expand All @@ -109,8 +115,11 @@ internal Expression ProjectAsWrapper(QueryBinderContext context, Expression sour
subContext.OrderByClauses = orderByClause.ToList();
}

if (isCollection)
if (isCollection
&& !isPrimitiveCollection
)
{

// new CollectionWrapper<ElementType> { Instance = source.Select(s => new Wrapper { ... }) };
return ProjectCollection(subContext, source, elementType, selectExpandClause, structuredType, navigationSource, orderByClause,
topOption,
Expand Down Expand Up @@ -232,7 +241,7 @@ public virtual Expression CreatePropertyValueExpression(QueryBinderContext conte

// Expression: source.Property
string propertyName = model.GetClrPropertyName(edmProperty);

PropertyInfo propertyInfo = source.Type.GetProperty(propertyName, BindingFlags.DeclaredOnly);
if (propertyInfo == null)
{
Expand All @@ -253,7 +262,7 @@ public class Child : Father
*/
propertyInfo = source.Type.GetProperties().Where(m => m.Name.Equals(propertyName, StringComparison.Ordinal)).FirstOrDefault();
}

Expression propertyValue = Expression.Property(source, propertyInfo);
Type nullablePropertyType = TypeHelper.ToNullable(propertyValue.Type);
Expression nullablePropertyValue = ExpressionHelpers.ToNullable(propertyValue);
Expand Down
Binary file not shown.
19 changes: 19 additions & 0 deletions test/Microsoft.AspNetCore.OData.E2E.Tests/Lists/ListsContext.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
using System;
using System.IO;
using Microsoft.EntityFrameworkCore;

anasik marked this conversation as resolved.
Show resolved Hide resolved
namespace Microsoft.AspNetCore.OData.E2E.Tests.Lists
{
public class ListsContext : DbContext
{
public ListsContext(DbContextOptions<ListsContext> options)
: base(options)
{

}

public DbSet<Product> Products{ get; set; }
public DbSet<Order> Orders{ get; set; }
}
}

186 changes: 186 additions & 0 deletions test/Microsoft.AspNetCore.OData.E2E.Tests/Lists/ListsController.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
//-----------------------------------------------------------------------------
// <copyright file="ListsController.cs" company=".NET Foundation">
// Copyright (c) .NET Foundation and Contributors. All rights reserved.
// See License.txt in the project root for license information.
// </copyright>
//------------------------------------------------------------------------------

using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.OData.Deltas;
using Microsoft.AspNetCore.OData.Formatter;
using Microsoft.AspNetCore.OData.Query;
using Microsoft.AspNetCore.OData.Routing.Attributes;
using Microsoft.AspNetCore.OData.Routing.Controllers;
using Microsoft.EntityFrameworkCore;
using Xunit;

namespace Microsoft.AspNetCore.OData.E2E.Tests.Lists
{
[Route("convention")]
public class ProductsController : ODataController
{
private readonly ListsContext _dbContext;
public ProductsController(ListsContext context)
{
_dbContext = context;
}

[EnableQuery(PageSize = 10, MaxExpansionDepth = 5)]
public IActionResult Get()
{
return Ok(_dbContext.Products);
}

[EnableQuery]
public IActionResult Get(string key)
{
return Ok(_dbContext.Products.Find(key));
}

public IActionResult Post([FromBody] Product Product)
{
Product.ProductId = _dbContext.Products.Count() + 1+"";
_dbContext.Products.Add(Product);
_dbContext.SaveChanges();

return Created(Product);
}

public IActionResult Put(int key, [FromBody] Product Product)
{
Product.ProductId = key+"";
Product originalProduct = _dbContext.Products.Find(key);

if (originalProduct == null)
{
_dbContext.Products.Add(Product);

return Created(Product);
}

_dbContext.Products.Remove(originalProduct);
_dbContext.Products.Add(Product);
return Ok(Product);
}

public IActionResult Patch(int key, [FromBody] Delta<Product> delta)
{
Product originalProduct = _dbContext.Products.Find(key);

if (originalProduct == null)
{
Product temp = new Product();
delta.Patch(temp);
_dbContext.Products.Add(temp);
return Created(temp);
}

delta.Patch(originalProduct);
return Ok(delta);
}

public IActionResult Delete(int key)
{
Product Product = _dbContext.Products.Find(key);

_dbContext.Products.Remove(Product);
return this.StatusCode(StatusCodes.Status204NoContent);
}

[HttpPost("ResetDataSource")]
public IActionResult ResetDataSource()
{
_dbContext.Orders.RemoveRange(_dbContext.Orders);
_dbContext.Products.RemoveRange(_dbContext.Products);
_dbContext.SaveChanges();

// Add new seed data
_dbContext.Products.AddRange(
new Product
{
ProductId = "1",
Name = "Product1",
Category = "Category1",
ListTestString = new List<string> { "Test1", "Test2", "Test99" },
ListTestBool = new List<bool> { true, false },
ListTestInt = new List<int> { 1, 2, 3 },
ListTestDouble = new List<double> { 1.1, 2.2 },
ListTestFloat = new List<float> { 1.1f, 2.2f },
ListTestDateTime = new List<DateTimeOffset> { DateTime.Now, DateTime.UtcNow },
ListTestUri = new List<Uri> { new Uri("https://example.com") },
ListTestUint = new uint[] { 1, 2, 3 },
ListTestOrder = new List<Order>
{
new Order { OrderId = "Order1" },
new Order { OrderId = "Order2" }
}
},
new Product
{
ProductId = "2",
Name = "Product2",
Category = "Category2",
ListTestString = new List<string> { "Test3", "Test4", "Test99" },
ListTestBool = new List<bool> { false, true },
ListTestInt = new List<int> { 4, 5, 6 },
ListTestDouble = new List<double> { 3.3, 4.4 },
ListTestFloat = new List<float> { 3.3f, 4.4f },
ListTestDateTime = new List<DateTimeOffset> { DateTime.Now.AddDays(1), DateTime.UtcNow.AddDays(1) },
ListTestUri = new List<Uri> { new Uri("https://example.org") },
ListTestUint = new uint[] { 4, 5, 6 }
},
new Product
{
ProductId = "3",
Name = "Product3",
Category = "Category3",
ListTestString = new List<string> { "Test5", "Test6" },
ListTestBool = new List<bool> { true, true },
ListTestInt = new List<int> { 7, 8, 9 },
ListTestDouble = new List<double> { 5.5, 6.6 },
ListTestFloat = new List<float> { 5.5f, 6.6f },
ListTestDateTime = new List<DateTimeOffset> { DateTime.Now.AddDays(2), DateTime.UtcNow.AddDays(2) },
ListTestUri = new List<Uri> { new Uri("https://example.net") },
ListTestUint = new uint[] { 7, 8, 9 }
},
new Product
{
ProductId = "4",
Name = "Product4",
Category = "Category4",
ListTestString = new List<string> { "Test98", "Test98" },
ListTestBool = new List<bool> { false, false },
ListTestInt = new List<int> { 10, 11, 12 },
ListTestDouble = new List<double> { 7.7, 8.8 },
ListTestFloat = new List<float> { 7.7f, 8.8f },
ListTestDateTime = new List<DateTimeOffset> { DateTime.Now.AddDays(3), DateTime.UtcNow.AddDays(3) },
ListTestUri = new List<Uri> { new Uri("https://example.edu") },
ListTestUint = new uint[] { 10, 11, 12 }
},
new Product
{
ProductId = "5",
Name = "Product5",
Category = "Category5",
ListTestString = new List<string> { "Test98", "Test98" },
ListTestBool = new List<bool> { true, false },
ListTestInt = new List<int> { 13, 14, 15 },
ListTestDouble = new List<double> { 9.9, 10.10 },
ListTestFloat = new List<float> { 9.9f, 10.10f },
ListTestDateTime = new List<DateTimeOffset> { DateTime.Now.AddDays(4), DateTime.UtcNow.AddDays(4) },
ListTestUri = new List<Uri> { new Uri("https://example.gov") },
ListTestUint = new uint[] { 13, 14, 15 }
}
);
_dbContext.SaveChanges();
return this.StatusCode(StatusCodes.Status204NoContent);
}


}

}
36 changes: 36 additions & 0 deletions test/Microsoft.AspNetCore.OData.E2E.Tests/Lists/ListsDataModel.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
//-----------------------------------------------------------------------------
// <copyright file="ListsDataModel.cs" company=".NET Foundation">
// Copyright (c) .NET Foundation and Contributors. All rights reserved.
// See License.txt in the project root for license information.
// </copyright>
//------------------------------------------------------------------------------

using System;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System.Runtime.Serialization;

namespace Microsoft.AspNetCore.OData.E2E.Tests.Lists
{
public class Product
{
[Key]
public string ProductId { get; set; }
public string Name { get; set; }
public string Category { get; set; }
public IList<string> ListTestString { get; set; } = new List<string>();
public IList<bool> ListTestBool { get; set; }
public IList<int> ListTestInt { get; set; }
public IList<double> ListTestDouble { get; set; }
public IList<float> ListTestFloat { get; set; }
public IList<DateTimeOffset> ListTestDateTime { get; set; }
public IList<Uri> ListTestUri { get; set; }
public uint[] ListTestUint { get; set; }
public IList<Order>? ListTestOrder { get; set; }
}
anasik marked this conversation as resolved.
Show resolved Hide resolved

public class Order
WanjohiSammy marked this conversation as resolved.
Show resolved Hide resolved
{
[Key] public string OrderId { get; set; }
}
}
29 changes: 29 additions & 0 deletions test/Microsoft.AspNetCore.OData.E2E.Tests/Lists/ListsEdmModel.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
//-----------------------------------------------------------------------------
// <copyright file="ListsEdmModel.cs" company=".NET Foundation">
// Copyright (c) .NET Foundation and Contributors. All rights reserved.
// See License.txt in the project root for license information.
// </copyright>
//------------------------------------------------------------------------------

using Microsoft.OData.Edm;
using Microsoft.OData.ModelBuilder;

namespace Microsoft.AspNetCore.OData.E2E.Tests.Lists
{
internal class ListsEdmModel
{
public static IEdmModel GetConventionModel()
{
ODataConventionModelBuilder builder = new ODataConventionModelBuilder();
EntitySetConfiguration<Product> Products = builder.EntitySet<Product>("Products");
EntitySetConfiguration<Order> Orders = builder.EntitySet<Order>("Orders");

builder.Namespace = typeof(Product).Namespace;
builder.Namespace = typeof(Order).Namespace;

var edmModel = builder.GetEdmModel();
return edmModel;
}

}
anasik marked this conversation as resolved.
Show resolved Hide resolved
}
Loading